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1Reachability Games and 
First-Order Logic 


1.1 Model Checking 


One of the fundamental algorithmic tasks in logic is model checking. 
For a logic L and a domain D of (finite) structures, the model-checking 
problem asks, given a structure 2 € D and a formula yp € L, whether 
2 is a model of wy. Notice that an instance of the model-checking 
problem has two inputs: a structure and a formula. We can measure 
the complexity in terms of both inputs, and this is what is commonly 
refered to as the combined complexity of the model-checking problem (for 
L and D). However, in many cases, one of the two inputs is fixed, and 
we measure the complexity only in terms of the other. If we fix the 
structure 2, then the model-checking problem for L on this structure 
amounts to deciding Th; (2) := {py ¢ L: A — wp}, the L-theory of A. 
The complexity of this problem is called the expression complexity of the 
model-checking problem (for L on 21). For first-order logic (FO) and for 
monadic second-order logic (MSO) in particular, such problems have a 
long tradition in logic and numerous applications in many fields. Of 
great importance in many areas of logic, in particular for finite model 
theory or databases, are model-checking problems for a fixed formula 
, which amounts to deciding the model class of p inside D, that is 
Modp() := {2 € D: AE wp}. Its complexity is the structure complexity 
or data complexity of the model-checking problem (for py on D). 


One of the important themes in this course is a game-based approach 
to model checking. The general idea is to reduce the problem whether 
A |= p toa strategy problem for a model checking game G(2,p) played 
by two players called Verifier (or Player 0) and Falsifier (or Player 1). We 
want to have the following relation between these two problems: 


AE yw iff Verifier has a winning strategy for G(2, ). 
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We can then do model checking by constructing, or proving the existence 


of, winning strategies. 


To assess the efficiency of games as a solution for model checking 
problems, we have to consider the complexity of the resulting model 
checking games based on the following criteria: 


e Are all plays necessarily finite? 

¢ If not, what are the winning conditions for infinite plays? 

¢ Do the players always have perfect information? 

¢ What is the structural complexity of the game graphs? 

¢ How does the size of the graph depend on different parameters of 
the input structure and the formula? 


For first-order logic (FO) and modal logic (ML) we have only fi- 
nite plays with positional winning conditions, and, as we shall see, 
the winning regions are computable in linear time with respect to the 
size of the game graph. Model checking games for fixed-point logics 
however admit infinite plays, and we use so-called parity conditions to 
determine the winner of such plays. It is still an open question whether 
winning regions and winning strategies in parity games are computable 


in polynomial time. 


1.2 Model Checking Games for Modal Logic 


The first logic that we discuss is propositional modal logic (ML). Let us 
first briefly review its syntax and semantics: 


Definition 1.1. Given a set A of actions and a set {P; : i € I} of atomic 
propositions, the set of formulae of ML is inductively defined: 


e All atomic propositions P; are formulae of ML. 
e If w, p are formulae of ML, then so are 7, (pA ¢) and (pV @g). 
e If » © MLanda€é A, then (a) € ML and [a] © ML. 


Remark 1.2. If there is only one action a € A, we write Ow and Lip 
instead of (a) and |a|w, respectively. 
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Definition 1.3. A transition system or Kripke structure with actions from a 
set A and atomic properties {P; : i € I} is a structure 


K = (V, (Ea)aca, (Piier) 


with a universe V of states, binary relations E, C V x V describing 
transitions between the states, and unary relations P; C V describing the 
atomic properties of states. 


A transition system can be seen as a labelled graph where the nodes 
are the states of K, the unary relations provide labels of the states, and 
the binary transition relations can be pictured as sets of labelled edges. 


Definition 1.4. Let K = (V,(Eq)aca,(Pi)icr) be a transition system, 
y € ML a formula and v a state of K. The model relationship K,v - y, 
ie., p holds at state v of K, is inductively defined: 


¢ K,v = P; if and only if v € Pj. 

e K,v — 7y if and only if K,v Fp. 

°K,vE wVv gifandonlyifK,v -porkK,vE @. 

°K,vE wg ifand only ifK,v — pandXK,v- g. 

¢ K,v — (a)y if and only if there exists w such that (v,w) € E, and 
K,wey. 

e K,v — [a] y if and only if K,w — wp holds for all w with (v,w) € Eq. 


For a transition system K and a formula w we define the extension 
y yp 
[y]* = {v: K,v - } as the set of states of K where p holds. 


For the game-based approach to model-checking, it is convenient to 
assume that modal formulae are written in negation normal form, i.e. 
negation is applied to atomic propositions only. This does not reduce 
the expressiveness of modal logic since every formula can be efficiently 
translated into negation normal form by applying De Morgan’s laws 
and the duality of UO and ¢ (i.e. =(a)p = [a]-p and |alp = (a)-1) to 
push negations to the atomic subformulae. 


Syntactically, modal logic is an extension of propositional logic. 
However, since ML is evaluated over transition systems, i.e. structures, 
it is often useful to see it as a fragment of first-order logic. 
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Theorem 1.5. For each formula y € ML there is a first-order formula 
y* (x) (with only two variables), such that for each transition system K 
and all its states v we have that K,v — p <== K — y*(v). 


Proof. The transformation is defined inductively, as follows: 


P; ++ P;x 
ap + ap" (x) 
(pop) ++ (p*(x) 0 p"(x)), where o € {A,V,—} 
(a)p —> Sy(Eaxy A p(y) 
[a]p > Vy(Eaxy > y*(y)) 


where *(y) is obtained from y*(x) by interchanging x and y every- 
where in the formula. Q.E.D. 


We are now ready to describe the model checking games for ML. 
Given a transition system K and a formula ~ € ML, we define a game 
G(K,w) whose positions are pairs (g,v) where @ is a subformula of 
y and v € V isa node of K. From any position (¢g,v) in this game, 
Verifier’s goal is to show that K,v = , whereas Falsifier tries to establish 
that K,v KF @. 

In the game, Verifier moves at positions of the form (9 V 0,7), with 
the choice to move either to (g,v) or to (0,7), and at positions ((a)9@,v), 
where she can move to any position (g,w) with w € vE,. Analogously, 
Falsifier moves from positions (gy A 0,v) to either (g,v) or (0,v), and 
from ([a|y,v) to any position (g,w) with w © vE,. Finally, at literals, 
ie. if p = P; or gp = —P,, the position (g,v) is a terminal position where 
Verifier has won if K,v — g, and Falsifier has won if K,v | 9. 

The correctness of the construction of G(K,w) follows readily by 


induction. 


Proposition 1.6. For any position (g,v) of G(K,w) we have that 


K,v Eo << Verifier has a winning strategy for G(K,) from (9,7). 


1.3 Reachability and Safety Games 


1.3 Reachability and Safety Games 


The model-checking games for propositional modal logic, that we have 
just discussed, are an instance of reachability games played on graphs or, 
more precisely, two-player games with perfect information and positional 
winning conditions, played on a game graph (or arena) 


G = (V, Vo, Vi-6) 


where the set V of positions is partitioned into sets of positions Vo and 
V, belonging to Player 0 and Player 1, respectively. Player 0 moves from 
positions v € Vo, while Player 1 moves from positions v € Vj. All moves 
are along edges, and so the interaction of the players, starting from an 
initial position vg, produces a finite or infinite play which is a sequence 
VpV102... with (v;,0;41) € E for all i. 

The winning conditions of the players are based on a simple posi- 
tional principle: Move or lose! This means that Player 7 has won at a 
position v in the case that position v belongs to his opponent and there 
are no moves available from that position. Thus the goal of Player ¢ is to 
reach a position in T, := {v € Vj_¢ : VE = ©}. We call this a reachability 
condition. 

But note that this winning condition applies to finite plays only. 
If the game graph admits infinite plays (for instance cycles) then we 
must either consider these as draws, or introduce a winning condition 
for infinite plays. The dual notion of a reachability condition is a safety 
condition where Player o just has the objective to avoid a given set of 
‘bad’ positions, which in this case is the set T;_,, and to remain inside 
the safe region V \ Ty_¢. 


A (positional) strategy for Player o in such a game G is a (partial) 
function f : {v € Vz: vE 4 ©} — V such that (v, f(v)) € E. A finite or 
infinite play vpv1v2... is consistent with f if vi41 = f(v;) for every i such 
that v; € V,. A strategy f for Player 7 is winning from vo if every play 
that starts at initial position vg and that is consistent with f is won by 
Player o. 


We first consider reachability games where both players play with the 
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reachability objective to force the play to a position in T;. We define 
winning regions 


Wo := {v € V: Player o has a winning strategy from position v}. 


If Wo UW, = V, i.e. for each v € V one of the players has a winning 
strategy, the game G is called determined. A play which is not won by 
any of the players is considered a draw. 


Example 1.7. No player can win from one of the middle two nodes: 


om—— 0 LE] 
Lee 


LO 


The winning regions of a reachability game G = (V, Vo, Vi, E) can 
be constructed inductively as follows: 


W? = T, and 
Witt — Wi {v € Ve: VENWL AO} U fv € Vi_g : VE C WE}. 


Clearly W! is the region of those positions from which Player 7 has a 
strategy to win in at most 7 moves, and for finite game graphs, with 
|V| =n, we have that W, = W?. 


Next we consider the case of a reachability-safety game, where Player 
0, as above, plays with the reachability objective to force the play to a 
terminal position in Tp, whereas player 1 plays with the safety objective 
of avoiding To, i.e. to keep the play inside the safe region S; := V \ To. 
Notice that there are no draws in such a game. 

The winning region Wo of Player 0 can be defined as in the case 
above, but the winning region W, of Player 1 is now the maximal set 
W C 5S, such that from all w € W Player 1 has a strategy to remain 
inside W, which can be defined as the limit of the descending chain 
W! DW] DW? D... with 


Ww? = S; and 
wit! — Win {ov © V: (v € Voand vE C Wi) or 
(v € Vj and vENW) 4 @)}. 
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Again on finite game graphs, with |V| = 1, we have that W; = WZ. 


This leads us to two fundamental concepts for the analysis of games 
on graphs: attractors and traps. Let G = (V, Vo, Vi, E) be a game graph 
and X C V. 


Definition 1.8. The attractor of X for Player o, in short Attr,(X) is the set 
of those positions from which Player o has a strategy to reach X (or to 
win because the opponent cannot move anymore). We can inductively 
define Attr(X) := Unen X", where 


X?S and 
X41 — XU {ve Ve: 0ENX' AO) U{v € Vy_g: VE C Xt}. 


For instance, the winning region W, in a reachability game is the 
attractor of the winning positions: Wo = Attr,(T,). 

A set Y CV \ Ti_¢ =: So is called a trap for Player 1 — ¢ if Player 7 
has a strategy to guarantee that from each v € Y the play will remain 
inside Y. Note that the complement of an attractor Attr,(X) is a trap 
for player 7. The maximal trap Y of Player 1 — ¢ can be defined as 
Y =(Mnen Y", where 


y =S, and 
yt! —y'q {v: (v € Vo and vENY' #@) or 
(v € Vj_, and vE C Y')}. 
The winning region of a Player 0 with the safety objective for S, is 
the maximal trap for player 1 — ¢. 


We consider several algorithmic problems for a given reachability 
game G: The computation of winning regions Wo and Wj, the computa- 
tion of winning strategies, and the associated decision problem 


GaME := {(G,v) : Player 0 has a winning strategy for G from v}. 


Theorem 1.9. Game is P-complete and decidable in time O(|V| + |E]). 


Note that this remains true for strictly alternating games. 
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Algorithm 1.1. A linear time algorithm for GAME 


Input: A game G = (V, Vo, Vi, E) 
output: Winning regions Wo and W; 


for all v € V do (* 1: Initialisation *) 
win[v] := L 
Plo] := {u: (u,v) € E} 
n|v] := |vE| 

end do 

for all v € Yo (* 2: Calculate win x) 


if n|v] = 0 then Propagate(v, 1) 
for all v € V; 

if n|v] = 0 then Propagate(v, 0) 
return win 


procedure Propagate(v, 7) 
if win|v] ~¢ L then return 


win[v] := 07 (* 3: Mark v as winning for player 7 *) 
for all u € P[v| do (« 4: Propagate change to predecessors *) 
nu] :=n[u] —1 
if u € V, or n[u] = 0 then Propagate(u, 7) 
end do 
end 


1.3 Reachability and Safety Games 


The inductive definition of an attractor shows that winning regions 
for both players can be computed efficiently. Hence we can also solve 
GAME in polynomial time. To solve GAME in linear time, we use the 
slightly more involved Algorithm 1.1. Procedure Propagate will be 
called once for every edge in the game graph, so the running time of 
this algorithm is linear with respect to the number of edges in G. 

Furthermore, we can show that the decision problem GAME is equiv- 
alent to the satisfiability problem for propositional Horn formulae. We 
recall that propositional Horn formulae are finite conjunctions /Aj-; C; of 
clauses C; of the form 


XyA...AXn > X or 
Xi A...A\ Xn — 0 
~ 


—— 


SY 
body(C;) head(C;) 


A clause of the form X or 1 + X has an empty body. 
We will show that Sat-Horn and GAME are mutually reducible via 
logspace and linear-time reductions. 


(1) GAME <jog-tin SAT-HORN 
For a game G = (V, Vo, V1, E), we construct a Horn formula g with 
clauses 


v—>u forall u € Vo and (u,v) € E, and 


V1 A...A0m 7 U for all u € Vi and uE = {v},..., 0m}. 


The minimal model of ig is precisely the winning region of Player 0, 
so 


(G,v) € GAME <= wg /(v— 0) is unsatisfiable. 
(2) SAT-HORN <iog-tin GAME 
For a Horn formula p(X1,...,Xn) = Aic) Ci, we define a game 


Gy = (V, Vo, Vi, E) as follows: 


V = {0} U{X1,..., Xn} U{C;: ie I} and 
_——. 1 —— 
0 1 
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E={X >C;: X = head(C;)} U{C; > X; : Xj € body(C;)}, 


i.e., Player 0 moves from a variable to some clause containing the 
variable as its head, and Player 1 moves from a clause to some 
variable in its body. Player 0 wins a play if, and only if, the play 
reaches a clause C with body(C) = ©. Furthermore, Player 0 has a 
winning strategy from position X if, and only if, ~ — X, so we have 


Player 0 wins from position0 <= > _ y is unsatisfiable. 


These reductions show that Sat-Horn is also P-complete and, in 
particular, also decidable in linear time. 


1.4 Games as an Algorithmic Construct: Alternating 
Algorithms 


Alternating algorithms are algorithms whose set of configurations is 
divided into accepting, rejecting, existential and universal configurations. 
The acceptance condition of an alternating algorithm A is defined by a 


game played by two players J and V on the computation graph G(A, x) 
(or equivalently, the computation tree 7(A,x)) of A on input x. The 
positions in this game are the configurations of A, and we allow moves 
C — C’ from a configuration C to any of its successor configurations 


C’. Player J moves at existential configurations and wins at accepting 
configurations, while Player V moves at universal configurations and 
wins at rejecting configurations. By definition, A accepts some input x if 


and only if Player 4 has a winning strategy for the game played on Ty y. 

We will introduce the concept of alternating algorithms formally, 
using the model of a Turing machine, and we prove certain relation- 
ships between the resulting alternating complexity classes and usual 
deterministic complexity classes. 


1.4.1 Turing Machines 


The notion of an alternating Turing machine extends the usual model of 
a (deterministic) Turing machine which we introduce first. We consider 
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Turing machines with a separate input tape and multiple linear work 
tapes which are divided into basic units, called cells or fields. Informally, 
the Turing machine has a reading head on the input tape and a com- 
bined reading and writing head on each of its work tapes. Each of the 
heads is at one particular cell of the corresponding tape during each 
point of a computation. Moreover, the Turing machine is in a certain 
state. Depending on this state and the symbols the machine is currently 
reading on the input and work tapes, it manipulates the current fields of 
the work tapes, moves its heads and changes to a new state. 


Formally, a (deterministic) Turing machine with separate input tape and 
k linear work tapes is given by a tuple M = (Q,T,&, qo, Face: Frej, 5), where 
Q is a finite set of states, X is the work alphabet containing a designated 
symbol 0 (blank), T is the input alphabet, gg € Q is the initial state, 
F := Facc U Frey © Q is the set of final states (with Facc the accepting states, 
Frey the rejecting states and Face Frei = ©), and 6 : (Q\ F) xT x LE > 
Q x {-1,0,1} x ZF x {-1,0,1} is the transition function. 


A configuration of M is a complete description of all relevant facts 
about the machine at some point during a computation, so it is a tuple 
C = (G,W1,...,Wk,X, Por Pty++++Pe) € QX (Z*)* x T* x N*+! where g 
is the current state, w; is the contents of work tape number 1, x is the 
contents of the input tape, po is the position on the input tape and 7; is 
the position on work tape number 7. The contents of each of the tapes 
is represented as a finite word over the corresponding alphabet|, i.e., a 
finite sequence of symbols from the alphabet]. The contents of each of 
the fields with numbers j > |w;| on work tape number i is the blank 
symbol (we think of the tape as being infinite). A configuration where x 
is omitted is called a partial configuration. The configuration C is called 
final if q € F. It is called accepting if q € Face and rejecting if q © Frej- 

The successor configuration of C is determined by the current state and 
the k + 1 symbols on the current cells of the tapes, using the transition 
function: If 5(4, X py, (W1) pyr- +1 (Wk) pp) = (9, M0, A, +++ Ak, M1, ..., Mg, b), 
then the successor configuration of C is A(C) = (q',a’, p’, x), where for 
any i, w is obtained from w; by replacing symbol number p; by a; and 
p = pi tm. We write C Fy C’ if, and only if, C’ = A(C). 

The initial configuration Co(x) = Co(M,x) of M on input x € I* is 


11 


1 Reachability Games and First-Order Logic 


given by the initial state qo, the blank-padded memory, i.e., w; = ¢ and 
pi — 0 for any i > 1, po = 0, and the contents x on the input tape. 

A computation of M on input x is a sequence Co, Ci,... of configu- 
rations of M, such that Co = Co(x) and C; Fy Cj41 for alli > 0. The 
computation is called complete if it is infinite or ends in some final con- 
figuration. A complete finite computation is called accepting if the last 
configuration is accepting, and the computation is called rejecting if the 
last configuration is rejecting. M accepts input x if the (unique) complete 
computation of M on x is finite and accepting. M rejects input x if the 
(unique) complete computation of M on x is finite and rejecting. The 
machine M decides a language L C I* if M accepts all x € L and rejects 
allx €I*\L. 


1.4.2 Alternating Turing Machines 


Now we shall extend deterministic Turing machines to nondeterministic 
Turing machines from which the concept of alternating Turing machines 
is obtained in a very natural way, given our game theoretical framework. 

A nondeterministic Turing machine is nondeterministic in the sense 
that a given configuration C may have several possible successor config- 
urations instead of at most one. Intuitively, this can be described as the 
ability to guess. This is formalised by replacing the transition function 
6:(Q\F) xT x Xk > Qx {-1,0,1} x Xx {-1,0,1}* by a transition 
relation A C ((Q\ F) xT x ZF) x (Q x {-1,0,1} x ZF x {-1,0,1}). 
The notion of successor configurations is defined as in the deterministic 
case, except that the successor configuration of a configuration C may 
not be uniquely determined. Computations and all related notions carry 
over from deterministic machines in the obvious way. However, on 
a fixed input x, a nondeterministic machine now has several possible 
computations, which form a (possibly infinite) finitely branching compu- 
tation tree 7)y,x. A nondeterministic Turing machine M accepts an input 
x if there exists a computation of M on x which is accepting, i.e., if there 
exists a path from the root Co(x) of Tiy4,x to some accepting configuration. 
The language of M is L(M) = {x € I*| M accepts x}. Notice that for 
a nondeterministic machine M to decide a language L C I™ it is not 
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necessary that all computations of M are finite. (In a sense, we count 
infinite computations as rejecting.) 

From a game-theoretical perspective, the computation of a nondeter- 
ministic machine can be viewed as a solitaire game on the computation 
tree in which the only player (the machine) chooses a path through the 
tree starting from the initial configuration. The player wins the game 
(and hence, the machine accepts its input) if the chosen path finally 
reaches an accepting configuration. 


An obvious generalisation of this game is to turn it into a two-player 


game by assigning the nodes to the two players who are called 4 and 


V, following the intuition that Player 4 tries to show the existence of a 
good path, whereas Player V tries to show that all selected paths are bad. 


As before, Player 4 wins a play of the resulting game if, and only if, the 
play is finite and ends in an accepting leaf of the game tree. Hence, we 


call a computation tree accepting if, and only if, Player 4 has a winning 
strategy for this game. 

It is important to note that the partition of the nodes in the tree 
should not depend on the input x but is supposed to be inherent to 
the machine. Actually, it is even independent of the contents of the 


work tapes, and thus, whether a configuration belongs to Player 4 or to 
Player V merely depends on the current state. 

Formally, an alternating Turing machine is a nondeterministic Turing 
machine M = (Q,T,%, qo, Face, Frej, 4) whose set of states Q = Q3U QyU 
Facc U Frej is partitioned into existential, universal, accepting, and rejecting 


states. The semantics of these machines is given by means of the game 
described above. 


Now, if we let accepting configurations belong to player V and 


rejecting configurations belong to player 3, then we have the usual 
winning condition that a player loses if it is his turn but he cannot move. 
We can solve such games by determining the winner at leaf nodes and 
propagating the winner successively to parent nodes. If at some node, 
the winner at all of its child nodes is determined, the winner at this 
node can be determined as well. This method is sometimes referred 
to as backwards induction and it basically coincides with our method 
for solving GAME on trees (with possibly infinite plays). This gives the 
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following equivalent semantics of alternating Turing machines: 
The subtree 7c of the computation tree of M on x with root C is 
called accepting, if 


¢ C is accepting 

* C is existential and there is a successor configuration C’ of C such 
that 7c is accepting or 

¢ C is universal and 7c is accepting for all successor configurations 
C’ of C. 


M accepts an input x, if To,(~) = Tm,x is accepting. 

For functions T,S : INN — N, an alternating Turing machine M is 
called T-time bounded if, and only if, for any input x, each computation 
of M on x has length less or equal T(|x|). The machine is called S- 
space bounded if, and only if, for any input x, during any computation 
of M on x, at most S(|x|) cells of the work tapes are used. Notice 
that time boundedness implies finiteness of all computations which is 
not the case for space boundedness. The same definitions apply for 
deterministic and nondeterministic Turing machines as well since these 
are just special cases of alternating Turing machines. These notions 
of resource bounds induce the complexity classes ATIME containing 
precisely those languages L such that there is an alternating T-time 
bounded Turing machine deciding L and Aspace containing precisely 
those languages L such that there is an alternating S-space bounded 
Turing machine deciding L. Similarly, these classes can be defined for 
nondeterministic and deterministic Turing machines. 

We are especially interested in the following alternating complexity 
classes: 


e ALoGSPACE = Ujgen ASPACE(d - logn), 
¢ APTIME = Ugen ATIME(11”), 
¢ APspace = Ugen ASPACE(11”). 


Observe that GAME € ALocspace. An alternating algorithm which 
decides GAME with logarithmic space just plays the game. The algorithm 
only has to store the current position in memory, and this can be done 
with logarithmic space. We shall now consider a slightly more involved 


example. 
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Example 1.10. QBF € AtimE(O(n)). W.l.o.g we assume that negation 
appears only at literals. We describe an alternating procedure Eval(,Z) 
which computes, given a quantified Boolean formula i and a valuation 
T : free(p) — {0,1} of the free variables of , the value []7. 


Algorithm 1.2. Alternating algorithm deciding QBF. 


Input: (~,Z) where p € QAL and T: free(p) > {0,1} 
if p= Y then 
if Z(Y) = 1 then accept 
else reject 
if ~ = 91 V 2 then ,,4” guesses i € {1,2} , Eval(@;,Z) 
if ~ = 91 A @2 then ,V” chooses i € {1,2}, Eval(@;,Z) 
(9, 
(9, 


ify = 1X@g _ then ,,” guesses j € {0,1} , Eval(g,Z|X = j]) 
ify =VX@g _ then ,V” chooses j € {0,1} , Eval(g,Z[X = j]) 


1.4.3 Alternating versus Deterministic Complexity Classes 


The main results we want to establish in this section concern the re- 
lationship between alternating complexity classes and deterministic 
complexity classes. We will see that alternating time corresponds to 
deterministic space, while by translating deterministic time into alternat- 
ing space, we can reduce the complexity by one exponential. Here, we 
consider the special case of alternating polynomial time and polynomial 
space. We should mention, however, that these results can be generalised 
to arbitrary large complexity bounds which are well behaved in a certain 


sense. 
Lemma 1.11. NPspace C APTIME. 


Proof. Let L € NPsprace and let M be a nondeterministic n!-space 
bounded Turing machine which recognises L for some ! € IN. The 
machine M accepts some input x if, and only if, some accepting config- 
uration is reachable from the initial configuration Co(x) in the config- 
uration tree of M on x in at most k := 2° steps for some c € N. This 
is due to the fact that there are most k different configurations of M on 
input x which use at most 1! cells of the memory which can be seen 
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using a simple combinatorial argument. So if there is some accepting 
configuration reachable from the initial configuration Co(x), then there 
is some accepting configuration reachable from C(x) in at most k steps. 
This is equivalent to the existence of some intermediate configuration C’ 
that is reachable from Co(x) in at most k/2 steps and from which some 
accepting configuration is reachable in at most */2 steps. 

So the alternating algorithm deciding L proceeds as follows. The ex- 
istential player guesses such a configuration C’ and the universal player 
chooses whether to check that C’ is reachable from Co(x) in at most */2 
steps or whether to check that some accepting configuration is reachable 
from C’ in at most k/2 steps. Then the algorithm (or equivalently, the 
game) proceeds with the subproblem chosen by the universal player, and 
continues in this binary search like fashion. Obviously, the number of 
steps which have to be performed by this procedure to decide whether 
x is accepted by M is logarithmic in k. Since k is exponential in n', the 
time bound of M is dn! for some d € IN, so M decides L in polynomial 
time. Q.E.D. 


Lemma 1.12. APTIME C PSPACE. 


Proof. Let L € APrime and let A be an alternating n!-time bounded 
Turing machine that decides L for some ! € IN. Then there is some 
r € IN such that any configuration of A on any input x has at most r 
successor configurations and w.l.o.g. we can assume that any non-final 
configuration has precisely r successor configurations. We can think of 
the successor configurations of some non-final configuration C as being 
enumerated as Cj,...,C;. Clearly, for given C and i we can compute C;. 
The idea for a deterministic Turing machine M to check whether some 
input x is in L is to perform a depth-first search on the computation tree 
Ta,x of A on x. The crucial point is that we cannot construct and keep 
the whole configuration tree 74 , in memory since its size is exponential 
in |x| which exceeds our desired space bound. However, since the length 
of each computation is polynomially bounded, it is possible to keep 
a single computation path in memory and to construct the successor 
configurations of the configuration under consideration on the fly. 
Roughly, the procedure M can be described as follows. We start 
with the initial configuration Co(x). Given any configuration C under 
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consideration, we propagate 0 to the predecessor configuration if C is 
rejecting and we propagate 1 to the predecessor configuration if C is 
accepting. If C is neither accepting nor rejecting, then we construct, 
fori = 1,...,r the successor configuration C; of C and proceed with 
checking Cj. If C is existential, then as soon as we receive 1 for some 1, 
we propagate 1 to the predecessor. If we encounter 0 for all i, then we 
propagate 0. Analogously, if C is universal, then as soon as we receive 
a 0 for some i, we propagate 0. If we receive only 1 for all 7, then we 
propagate 1. Then x is in L if, and only if, we finally receive 1 at Co(x). 
Now, at any point during such a computation we have to store at most 
one complete computation of A on x. Since A is n!-time bounded, each 
such computation has length at most 1! and each configuration has size 


21 


at most c-n! for some c € N. So M needs at most c- n*! memory cells 


which is polynomial in n. Q.E.D. 
So we obtain the following result. 


Theorem 1.13. (Parallel time complexity = sequential space complexity) 


(1) APTIME = PSPAceE. 
(2) AEXPTIME = EXPSPACE. 


Proposition (2) of this theorem is proved exactly the same way as 
we have done it for proposition (1). Now we prove that by translating 
sequential time into alternating space, we can reduce the complexity by 
one exponential. 


Lemma 1.14. ExpTimME C APSPACE 


Proof. Let L € Exprime. Using a standard argument from complexity 
theory, there is a deterministic Turing machine M = (Q,%,q9,6) with 
time bound m := 2°". for some c, k € IN with only a single tape (serving 
as both input and work tape) which decides L. (The time bound of 
the machine with only a single tape is quadratic in that of the original 
machine with k work tapes and a separate input tape, which, however, 
does not matter in the case of an exponential time bound.) Now if 
T =X (Q x X) w {#}, then we can describe each configuration C of M 
by a word 
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C = #wo... Wj_-1(quw;)Wi41... wet € I™. 


Since M has time bound m and only one single tape, it has space bound 
m. So, w.Lo.g., we can assume that |C| = m +2 for all configurations C of 
M on inputs of length n. (We just use a representation of the tape which 
has a priori the maximum length that will occur during a computation 
on an input of length n.) Now the crucial point in the argumentation is 
the following. If CF C’ and 1 < i < m, symbol number i of the word 
C’ only depends on the symbols number i — 1, i and i+ 1 of C. This 
allows us to decide whether x € L(M) with the following alternating 
procedure which uses only polynomial space. 


Player 4 guesses some number s < m of steps of which he claims 
that it is precisely the length of the computation of M on input x. 


Furthermore, 4 guesses some state q € Fac, a Symbol a € X% and 
a number i € {0,...,s}, and he claims that the i-th symbol of the 
configuration C of M after the computation on x is (qa). (So players 
start inspecting the computation of M on x from the final configuration.) 


If M accepts input x, then obviously player 3 has a possibility to choose 
all these objects such that his claims can be validated. Player V wants to 


disprove the claims of 4. Now, player 4 guesses symbols a_1, 49,4, € T 
of which he claims that these are the symbols number i — 1,7 andi+1 
of the predecessor configuration of the final configuration C. Now, 


VY can choose any of these symbols and demand that J validates his 
claim for this particular symbol. This symbol is now the symbol under 
consideration, while i is updated according to the movement of the 
(unique) head of M. Now, these actions of the players take place for 
each of the s computation steps of M on x. After s such steps, we check 
whether the current symbol and the current position are consistent with 
the initial configuration Co(x). The only information that has to be 
stored in the memory is the position 7 on the tape, the number s which 


4 has initially guessed and the current number of steps. Therefore, the 
algorithm uses space at most O(log(m)) = O(n") which is polynomial in 


n. Moreover, if M accepts input x then obviously player 4 has a winning 
strategy for the computation game. If, conversely, M rejects input x, then 


the combination of all claims of player 4 cannot be consistent and player 
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V has a strategy to spoil any (cheating) strategy of player 4 by choosing 
the appropriate symbol at the appropriate computation step. Q.E.D. 


Finally, we make the simple observation that it is not possible 
to gain more than one exponential when translating from sequential 
time to alternating space. (Notice that ExprimE is a proper subclass of 
2EXPTIME.) 


Lemma 1.15. APspace C ExPTIME 


Proof. Let L € APspact, and let A be an alternating n‘-space bounded 
Turing machine which decides L for some k € IN. Moreover, for an input 
x of A, let Conf(A, x) be the set of all configurations of A on input x. Due 
to the polynomial space bound of A, this set is finite and its size is at most 
exponential in |x|. So we can construct the graph G = (Conf(A,x),F) 
in time exponential in |x|. Moreover, a configuration C is reachable from 
Co(x) in T4,x if and only if C is reachable from Co(x) in G. So to check 
whether A accepts input x we simply decide whether player 4 has a 


winning strategy for the game played on G from C(x). This can be 
done in time linear in the size of G, so altogether we can decide whether 
x € L(A) in time exponential in |x]. Q.E.D. 


Theorem 1.16. (Translating sequential time into alternating space) 
(1) ALocspace = P. 


(2) APSPACE = EXPTIME. 


Proposition (1) of this theorem is proved using exactly the same 
arguments as we have used for proving proposition (2). An overview 
over the relationship between deterministic and alternating complexity 
classes is given in Figure 1.1. 


LOGSPACE © PTIME C Pspace C ExpTIME C EXpPsPAcE 


ALocspace C APtimE C APspace C AEXPTIME 


Figure 1.1. Relation between deterministic and alternating complexity classes 
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1.5 Model Checking Games for First-Order Logic 


Let us first recall the syntax of FO formulae on relational structures. 
We have that R;(Z), —R;(X), x = y and x ¥ y are well-formed valid FO 
formulae, and inductively for FO formulae g and w, we have that 9 V y, 


gp \ p, Ix@ and Vx@ are well-formed FO formulae. This way, we allow 
only formulae in negation normal form where negations occur only at 
atomic subformulae and all junctions except V and / are eliminated. 
These constraints do not limit the expressiveness of the logic, but the 
resulting games are easier to handle. 

For a structure % = (A,R1,...,Rm) with R; C A", we define the 
evaluation game G(2, w) as follows: 

We have positions g(a) for every subformula ¢(%) of p and every 
ae AX, 


At a position @ V 0, Verifier can choose to move either to @ or to 


8, while at positions 4x@(x,b), he can choose an instantiation a € A of 
x and move to (a,b). Analogously, Falsifier can move from positions 
g \ 8 to either g or 8 and from positions Vx@(x,b) to g(a,b) for an 
we As 

The winning condition is evaluated at positions with atomic or 
negated atomic formulae ¢, and we define that Verifier wins at g(a) if, 
and only if, 21 — g(a), and Falsifier wins if, and only if, 2 FF g(a). 

In order to determine the complexity of FO model checking, we 
have to consider the process of determining whether 2 | . To decide 
this question, we have to construct the game G(2l,) and check whether 
Verifier has a winning strategy from position . The size of the game 
graph is bound by |G(2,~)| < |p| -|A|v4"), where width() is the 
maximal number of free variables in the subformulae of . So the game 
graph can be exponential, and therefore we can get only exponential time 
complexity for GAME. In particular, we have the following complexities 
for the general case: 


e alternating time: O(|#| + qd(w) log | A|) 

where qd(w) is the quantifier-depth of y, 
e alternating space: O(width(w) - log |A| + log |y]), 
¢ deterministic time: O(|| - |A|“4#) and 
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¢ deterministic space: O(|p| + qd(q) log |A]). 


Efficient implementations of model checking algorithms will con- 
struct the game graph on the fly while solving the game. 

We obtain that the structural complexity of FO model checking 
is ALOcTIME, and both the expression complexity and the combined 
complexity are PSPACE. 


Fragments of FO with Efficient Model Checking 

We have seen that the size of the model checking games for first- 
order fomulae is exponential with respect to the width of the formulae, 
so we do not obtain polynomial-time model-checking algorithms in the 
general case. We now consider appropriate restrictions of FO, that lead 
to fragments with small model-checking games and thus to efficient 
game-based model-checking algorithms. 

The k-variable fragment of FO is 


FOF := { € FO: width(p) < k}. 


Clearly |G(2,)| < |p| -|A|* for any finite structure % and any 
p € FO. 


Theorem 1.17. ModCheck(FO*) is solvable in time O(|y| - |A|‘) and 
P-complete, for every k > 2. 


As shown in Theorem 1.5, modal logic can be embedded (efficiently) 
into FO?. Hence, also ML model checking has polynomial time complex- 
ity. 

It is a general observation that modal logics have many convenient 
model-theoretic and algorithmic properties. Besides efficient model- 
checking the following facts are important in many applications of 
modal logic. 


¢ The satisfiability problem for ML is decidable (in PSPAcE), 

¢ ML has the finite model property: each satisfiable formula has a 
finite model, 

¢ ML has the tree model property: each satisfiable formula has a 
tree-shaped model, 
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¢ algorihmic problems for ML can be solved by automata-based meth- 
ods. 


The embedding of ML into FO? has sometimes been proposed as 
an explanation for the good properties of modal logic, since FO? is a 
first-order fragment that shares some of these properties. However, more 
recently, it has been seen that this explanation has its limitations and is 
not really convincing. In particular, there are many extensions of ML 
to temporal and dynamic logics such as LTL, CTL, CTL*, PDL and the 
p-calculus L, that are of great importance for applications in computer 
science, and that preserve many of the good algorithmic properties of 
ML. Especially the associated satisfiability problems remain decidable. 
However this is not at all true for the corresponding extension of FO. 

A better and more recent explanation for the good properties of 
modal logic is that modal operators correspond to a restricted form of 
quantification, namely guarded quantification. Indeed, in the embedding 
of ML into FO, all quantifiers are guarded by atomic formulae. This can 
be vastly generalised beyond two-variable logic and involving formulae 
of arbitrary relational vocabularies, leading to the guarded fragment of 
FO. 


Definition 1.18. The guarded fragment of first-order logic GF is the frag- 
ment of first-order logic which allows only guarded quantification 


Ay(a(<, 9) A p(Z,¥)) and Vi(a(Z,9) > 9(%9)), 


where the guards « are atomic formulae containing all free variables of @. 


GF is a generalisation of modal logics: ML C GF C FO. Indeed, the 
modal operators } and L) can be expressed as 


(a)@ = Sy(Eaxy A p(y)) and [ale = Vy(Eaxy > ¢(y)). 


It has turned out that the guarded fragment preserves (and explains 
to some extent) essentially all of the good model-theoretic and algorith- 
mic properties of modal logics, in a far more expressive setting. In terms 


of model-checking games, we can observe that guarded logics have small 
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model checking games of size ||G(2l, p) || = O(|y| - ||2l||), and so there 
exist efficient game-based model-checking algorithms for them. 


px) 


2Parity Games and Fixed-Point 
Logics 


In the first chapter we have discussed model checking games for first- 
order logic and modal logic. These games admit only finite plays and 
their winning conditions are specified just by sets of positions, that the 
players want to reach. Winning regions in these games can be computed 
in linear time with respect to the size of the game graph. 

However, in many computer science applications, more expressive 
logics are needed, such as temporal logics, dynamic logics, fixed-point 
logics and others. Model checking games for these logics admit infinite 
plays and their winning conditions must be specified in a more elaborate 
way. As a consequence, we have to consider the theory of infinite games. 

For fixed-point logics, such as LFP or the modal p-calculus, the 
appropriate evaluation games are parity games. These are games of 
possibly infinite duration with a function that assigns to each position 
a natural number, called its priority. The winner of an infinite play is 
determined according to whether the least priority seen infinitely often 
during the play is even or odd. 


2.1 Parity Games 


Definition 2.1. A parity game is given by a labelled game graph G = 
(V, Vo, Vi, E,Q.) as in Sect. 1.3 with a function O: V > IN that assigns a 
priority to each position. The set V of positions may be finite or infinite, 
but |O(V)|, the number of different priorities which is called the index 
of G, must be finite. As before, a finite play is lost by the player who 
gets stuck, i.e. cannot move. For infinite plays v9v102..., we have the 
parity winning condition: If the least number appearing infinitely often in 
the sequence (v9)O.(v1) ... of priorities is even, then Player 0 wins the 
play, otherwise Player 1 wins. 

A strategy (for Player 7) is a function f : V*V, — V such that 
f(U001...Un) © UnE. We say that a play 77 = v9v 1... is consistent with the 
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strategy f of Player o if for each v; € Vz it holds that vj, = f(v0...v;). 
The strategy f is winning for Player o from (or on) a set W C V if each 
play starting in W that is consistent with f is won by Player oc. 


In general, a strategy may depend on the entire history played so far, 
and can thus be a very complicated object. However, we are interested 
in simple strategies that depend only on the current position. 


Definition 2.2. A strategy (of Player c) is called positional (or memoryless) 
if it only depends on the current position, but not on the history of the 
play, which means that f(hv) = f(h'v) for all h,h’ € V*,v € V. We can 
view positional strategies simply as functions f : V, > V. 


We shall see that positional strategies suffice to solve parity games. 
Before we formulate and prove this Forgetful Determinacy Theorem, we 
recall that positional strategies are of course sufficient whenever, as in 
the previous chapter, the players have purely positional objectives such 
as reachability or safety. Specifically, for every game G = (V, Vo, Vi, E) 
and every X C V we have defined the attractor 


Attro(X) = {v € V: Player o has a strategy from v to reach some 
position x € X UT,} 


and such an attractor strategy can, without loss of generality, assumed 
to be positional. Similarly, if Y C V is a trap for Player o, then Player 
(1 —¢) has a positional trap strategy to keep the play inside Y. 

Further we note that positional winning strategies on parts of the 
game graph may be combined to positional winning strategies on larger 
regions. Indeed, let f and f’ be positional strategies for Player ¢ that 
are winning exactly on the sets W, W’, respectively. Let (f + f’) be the 
positional strategy defined by 


f(x) ifxew 


f'(x) otherwise. 


(f+ FV) = 


Then (f + f’) is a winning strategy on WU W’. 
We can now turn to the proof of the Forgetful Determinacy Theorem. 
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Theorem 2.3 (Forgetful Determinacy). In any parity game, the set of 
positions can be partitioned into two sets Wo and W, such that Player 0 
has a positional strategy that is winning on Wo and Player 1 has a 
positional strategy that is winning on W). 


Proof. Let G = (V,Vo,Vi,E,Q) be a parity game with |O(V)| = m. 
Without loss of generality we can assume that O(V) = {0,...,m—1} or 
O(V) = {1,...,m}. We prove the statement by induction over |O(V)|. 
In the case that |O.(V)| = 1, ie, A(V) = {0} or O(V) = {1}, either 
Player 0 or Player 1 wins every infinite play. Her opponent can only win 


by reaching a terminal position that does not belong to him. So we have, 
for O(V) = {c}, 


Wi_¢ = Attry_¢(Ti_<) and 
We =V\Wi-o. 


Computing W,_, as the attractor of T;_, is a simple reachability prob- 
lem, and thus it can be solved with a positional strategy. For We there is 
a positional strategy that avoids leaving this (1 — o)-trap. 

Let now |Q(v)| = m > 1. We explicitly consider the case that 
0 € AV), ie, O(V) = {0,...,m—1}. Otherwise, if the minimal 
priority is 1, we can use the same argumentation with switched roles of 
the players. We define 


X, := {v € V: Player 1 has a positional winning strategy from v}, 


and let g be a positional winning strategy for Player 1 on Xj. 
Our goal is to provide a positional winning strategy f* for Player 0 
on Xo := V \ Xj, so in particular we have W, = X; and Wo = V \ Xj. 
First of all, observe that Xo is a trap for Player 1. Indeed, if Player 1 
could reach X; from some v € Xo, then Player 1 could win with a 
positional strategy from v, so v would also be in X;. Thus, there exists a 
positional trap strategy t for Player 0 on Xo that guarantees that a play 


remains inside Xo. 


Let Y = Q71(0) M Xp and Z = Attro(Y). Player 0 has a positional 
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attractor strategy a to ensure, from every position z € Z \ Y, that Y (ora 
terminal winning position in To) is reached in finitely many steps. 

Let now V’ = V \ (X, UZ). The restricted game G’ = G|y has 
strictly fewer priorities than G (since at least all positions with priority 
0 have been removed). Thus, by induction hypothesis, the Forgetful 
Determinacy Theorem holds for G’. This means that V’ = Wj U W; and 
there exist positional winning strategies f’ for Player 0 on W6 and g’ for 
Player 1 on W in G’. 

However, it follows that W; = @, since the strategy 


g(x) xExy 
(gtg)ixre ge” 
g(x) xEew, 


is a positional winning strategy for Player 1 on X; U W}. Indeed, every 
play consistent with (g + g’) either stays in Wj and is consistent with 
g’ or reaches X; and is from this point on consistent with g. But X1, by 
definition, already contains all positions from which Player 1 can win 
with a positional strategy, so W; = ©. 


X1 


a77*(0) 


\ 


Figure 2.1. Construction of a winning strategy 
Knowing that W; = @, let f* = f’+a+t, ie. 


f'(x) if x © WG 
fi(x)= a(x) ifxeZ\Y 
t(x) ifxeY 


We claim that f* is a positional winning strategy for Player 0 from Xo. 
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Note that if 7t is a play that is consistent with f*, then 7t remains inside 
Xo. We distinguish two cases. 

Case (a): 7 hits Z only finitely often. Then 7r eventually stays in Wj and 
is consistent with f’ from this point onwards. Hence Player 0 wins 77. 
Case (b): 7 hits Z infinitely often. Then 7 also hits Y infinitely often, 
which implies that priority 0 is seen infinitely often. Thus, Player 0 


wins 71. Q.E.D. 
The following theorem is a consequence of positional determinacy. 


Theorem 2.4. It can be decided in NPM coNP whether a given position 
in a parity game is a winning position for Player 0. 


Proof. A node v in a parity game G = (V,V0,V1,E,Q) is a winning 
position for Player o if there exists a positional strategy f : V; — V 
which is winning from position v. It therefore suffices to show that the 
question whether a given strategy f : V, — V is a winning strategy for 
Player o from position v can be decided in polynomial time. We prove 
this for Player 0; the argument for Player 1 is analogous. 

Given G and f : Vo + V, we obtain a reduced game graph Gy = 
(W, F) by retaining only those moves that are consistent with f, ie., 


F={(v,w):(veWNV,Aw= f(v)) Vv 
(vEWNV_, A (v,w) € EJ}. 


In this reduced game, only the opponent, Player 1, makes non-trivial 
moves. We call a cycle in (W, F) odd if the least priority of its nodes 
is odd. Clearly, Player 0 wins G from position v via strategy f if, and 
only if, in Gf no odd cycle and no terminal position w € Vo is reachable 
from v. Since the reachability problem is solvable in polynomial time, 
the claim follows. Q.E.D. 


2.2 Algorithms for parity games 


It is an open question whether winning sets and winning strategies for 
parity games can be computed in polynomial time. The best algorithms 
known today are polynomial in the size of the game, but exponential 
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with respect to the number of priorities. On an class of parity games 
with bounded index, such algorithms run in polynomial time. 

One way to intuitively understand an algorithm solving a parity 
game is to imagine a referee who watches the players playing the game. 
At some point, the referee is supposed to say “Player 0 wins”, and 
indeed, whenever the referee does so, there should be no question that 
Player 0 wins. We shall first give a formal definition of a certain kind 
of referee with bounded memory, and later use this notion to construct 
algorithms for parity games. 


Definition 2.5. A referee M = (M,mo,6,F) for a parity game G = 
(V, Vo, Vi, E,O) consists of a set of states M with a distinguished initial 
state mo € M, a set of final states F C M, and a transition function 
6: VxM-— M. Note that a referee is thus formally the same as an 
automaton reading words over the alphabet V. But to be called a referee, 
two further conditions must be satisfied, for any play vv; ... of G, and 
and the corresponding sequence mom}... of states of M, where mpo is 
the initial state of M and mj41 = 6(v;, mj): 

(1) If vg... is winning for Player 0, then there is a k such that m, € F, 

(2) If m, € F for some k, then there exist i <j < k such that v; = v; and 

min{O( 041), O(0j42),---,O(v;) } is even. 

To illustrate the second condition in the above definition, note that 
in the play vpv1... the sequence ¥;0;,;...v; forms a cycle. Assuming 
that both players use a positional strategy the decision of the referee is 
correct. Indeed, if a cycle with even priority appears, then this cycle will 
be repeated forever, Player 0 can be declared as the winner. To capture 
this intuition formally, we define the following reachability game, which 
emerges as the product of the original game G and the referee M. 


Definition 2.6. Let G = (V,Vo, V1, E,Q) be a parity game and M = 
(M,mo,6,F) an automaton reading words over V. We associate with G 
and M a reachability game 


Gx M=(VxM,VxM,V, x M,E',V x F), 


where ((v,m), (v’,m’)) € E’ iff (v,v’) € E and m' = 6(v,m), and V x F 
is the set of positions which are immediately winning for Player 0 (the 
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goal of Player 0 is to reach such a position). Plays that do not reach a 
position in V x F are won by Player 1. 


Note that M in the definition above is a deterministic automaton, 
i.e., 0 is a function. Therefore, in G and in G x M the players have the 
same choices, and thus it is possible to translate strategies between G 
and G x M. Formally, for a strategy f in G we define the strategy f in 
G x Mas 


f((V0,mMo) (01,111)... (Un, Mn)) = (f (V001--- Un), 6(On, Mn)).- 


Conversely, given a strategy f in G x M we define the strategy f in G 
such that f(v001 ...0n) = Un4i if and only if 


f((Go, mo) (V1, 111)... (On, ttn) = (Unga, Mn+1), 


where mom... is the unique sequence corresponding to v9v .... 
With G x M we are ready to prove that the definition of a referee 
indeed makes sense for parity games. 


Theorem 2.7. Let G be a parity game and ™ a referee for G. Then 
Player 0 wins G from vp if, and only if, she wins G x M from (v9, M9). 


Proof. (=) Let f be a winning strategy for Player 0 in G from vp. Assume 
that Player 0 does not have a winning strategy for G x M from (v9, m0). 
By determinacy of reachability games, there exists a winning strategy g 
for Player 1. Consider the unique play 71g = vov,... that is consistent 
with f and g and the unique play 71g, = (¥0,mo) (01,11)... which 
is consistent with f and g. Observe that the positions of G appearing 
in both plays are indeed the same due to the way f and g are defined. 
Since Player 0 wins 7tg, by Property (1) in the definition of a referee 
there must be an m, € F. But this contradicts the fact that Player 1 wins 
MGxM- 

(<=) Let f be a winning strategy for Player 0 in G x M, and assume 
that Player 1 has a positional winning strategy g in G. Again, we consider 
the unique plays pig = vov,... and 71gxy = (V0,Mo)(V1,m)... such 
that 7tg is consistent with f and g, and 7tgx \y is consistent with f and 
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g. Since 7tgx\4 is won by Player 0, there is an m, € F appearing in this 
play. 

By Property (2) in the definition of a referee, there exist two indices 
i <j such that v; = v; and the minimum priority appearing between 1; 
and v; is even. Let us now consider the following strategy f’ for Player 0 


in G: 


; _ Jf(wowr...Wn) ifn <j, 
f (Wow 1...Wn) = 
f(WoW,...Wm) otherwise, 


where m = i+ |[(n—i) mod (j —i)]. Intuitively, the strategy f’ makes 
the same choices as f up to the (j — 1)st step, and then repeats the 
choices of f from steps Pa eer atl 

We claim that the unique play 7’ in G that is consistent with both 
f’ and g is won by Player 0. Since in the first j steps f’ is the same as f, 
we have that [7] = 0, for all n < j. Now observe that 7[j + 1] = 0)41. 


Since g is positional, if v; is a position of Player 1, then 7[j +1] = vj41, 
and if v; is a position of Player 0, then 7t[j + 1] = v;41 because we defined 
f'(vo--.0;) = f(vo--.0;). By induction we get that the play 7 repeats 
the cycle vjv;,1 ...0; infinitely often, ie. 


7T = U0... Vj_-1(U;Vi41 Cras Opals 


Thus, the minimal priority occurring infinitely often in 7 is the same as 
min{O(v;), O(0j41),...O(o;-1) }, and thus is even. Therefore Player 0 
wins 7t, which contradicts the fact that g was a winning strategy for 
Player 1. Q.E.D. 


This theorem allows us, if a referee is known, to reduce the problem 
of solving a parity game to the problem of solving a reachability game, 
which we already tackled with the GAME algorithm. But to make use of 
it, we first need to construct a referee for a given parity game. 

The most naive way to build a referee for a parity game is to just 
remember, for each position v visited during the play, the minimal 
priority seen since the last occurrence of v. If it happens that a position 


o2, 
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v is repeated, and the minimal priority seen since the last occurrence of 
v is even, the referee decides that Player 0 wins the play. 

It is easy to check that an automaton defined in this way is indeed a 
referee for G, but such a referee can be very big. Since for each of the 
|V| =n positions we need to store one of |O(V)| = d colours, the size 
of the referee is in the order of O(d"). We shall present a referee that is 
much better for small d. 


Definition 2.8. A progress-measuring referee Mp = (Mp, mo, op, Fp) for 
a parity game G = (V,Vo,Vj,E,Q) is constructed as follows. If n; = 
|Q~1(i)| is the number of positions with priority i, then 


Mp = {0,1,...,m9 +1} x {0} x {0,1,...,m2 +1} x {O} x... 


and this product ends in --- x {0,1,...,%m +1} if the maximal priority 
m is even, or in --- x {0} if itis odd. The initial state is mp = (0,...,0), 
and the transition function 6(v,C¢) with ¢ = (co,0,c2,0,...,Cm) is given 


by 


5(v,2) (Gy0; 05, Oh% “tatey  1,0;0,+- .,0) if O(v) is even, 

vo 

Co, 0,C2,0,...,€ _,,0,0,0,...,0 otherwise. 
Q(v)-1 


The set Fp contains all tuples (cg,0,c2,...,Cm) in which some counter 
cj = nj +1 reached the maximum possible value. 


The intuition behind Mp is that it counts, for each even priority p, 
how many positions with priority p were seen without any lower priority 
in between. If more than np such positions are seen, then at least one 
must have been repeated, which guarantees that Mp is a referee. 


Lemma 2.9. For each finite parity game G the automaton  p constructed 
above is a referee for G. 


Proof. We need to show that Mp exhibits the two properties characteris- 
ing a referee: 


(1) if v9... is winning for Player 0, then there is a k such that m, € F, 
(2) if, for some k, my € F, then there exist 1 <j < k such that 0; = 9; 
and min{OQ(0;+1),O(042),-..,O(v;)} is even. 
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To see (1), assume that v9v1... is a play winning for Player 0. Let 
k be such an index that O(v,) is even, appears infinitely often in 
O(0,)O(0¢41)-.., and no priority lower than O(v;) appears in this 
play suffix. Then, starting from v;, the counter cq,,) will never be 
decremented, but it will be incremented infinitely often. Thus, for a 
finite game G, it will reach nqip,) + 1 at some point, ie. a state in Fp. 
To prove (2), let vov1 ... vx, be such a prefix of a play that after v, 
some counter Cy is set to Np) + 1 for an even priority p. Let v;, be the last 
position at which this counter was 0, and v;,, the subsequent positions 
at which it was incremented, up to i, = k. All positions 0;),0;,,..., Vinp 
have priority p, but since there are only n, different positions with 
priority p, we get that, for some k < I, 0;, = v;,. Now i, and {; are the 
positions required to witness (2), because indeed the minimum priority 
between i; and 7; is p since cp was not reset in between. Q.E.D. 


For a parity game G with an even number of priorities d, the above 
presented referee has size 1g - Nz - - - Ng_2, which is at most (a5) ”?. We 
get the following corollary. 


n 


Corollary 2.10. Parity games can be solved in time O(m - (a95)%/ i 


Notice that the algorithm using a referee has high space demand: 
Since the product game G x Mp must be explicitly constructed, the space 
complexity of this algorithm is the same as its time complexity. There is a 
method to improve the space complexity by storing the maximal counters 
the referee Mp uses in each position and lifting such annotations. This 
method is called game progress measures for parity games. We will not 
define it here, but the equivalence to modal p-calculus proven in the 
next chapter will provide another algorithm for solving parity games 
with polynomial space complexity. 


2.3 Fixed-Point Logics 


We will define two fixed-point logics, the modal p-calculus, L,, and the 
first-order least fixed-point logic, LFP, which extend modal logic and 
first-order logic, respectively, with the operators for least and greatest 
fixed-points. 
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The syntax of L, is analogous to modal logic, with two additional 
rules for building least and greatest fixed-point formulas: 


uX.p(X) and vX.~(X) 


are L, formulas if p(X) is, where X is a variable that can be used in @ 
the same way as predicates are used, but must occur positively in @, i.e. 
under an even number of negations (or, if g is in negation normal form, 
simply non-negated). 

The syntax of LFP is analogous to first-order logic, again with two 
additional rules for building fixed-points, which are now syntactically 
more elaborate. Let p(T, x1,X2,...Xn) be a LFP formula where T stands 
for an n-ary relation and occurs only positively in g. Then both 


[Ifp Tx.p(T, X)|(@) and [gfp Tx.p(T, x)| (a) 


are LFP formulas, where @ = 4)... ay. 

To define the semantics of L, and LFP, observe that each formula 
p(X) of Ly or g(T,*) of LFP defines an operator [g(X)] : P(V) > 
P(V) on states V of a Kripke structure K and [g(T,x)] : P(A") > 
P(A") on tuples from the universe of a structure 2. The operators 
are defined in the natural way, mapping a set (or relation) to a set or 
relation of all these elements, which satisfy g with the former set taken 


as argument: 
lp(X)(B) = tv € KR: K,0 F e(B)}, and 


[p(T X)](R) = {a € A: AE P(R,a)}. 

An argument B is a fixed-point of an operator f if f(X) = X, and to 
complete the definition of the semantics, we say that 1.X.p(X) defines 
the smallest set B that is a fixed-point of [g(X)], and vX.~(X) defines the 
largest such set. Analogously, |lfp Tx.~(T, X)|(*) and |gfp Tx.p(T, x)](*) 
define the smallest and largest relations being a fixed-point of [g(T,x)]], 
respectively. In a few paragraphs, we will give an alternative characteri- 
sation of least and greatest fixed-points, which is better tailored towards 
an algorithmic computation. 
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To justify this definition, we have to assure that all notions are well- 
defined, i.e., in particular, we have to show that the operators actually 
have fixed-points, and that least and greatest fixed-points always exist. 
In fact, this relies on the monotonicity of the operators used. 


Definition 2.11. An operator F is monotone if 
RGY SFX) CE )s 


The operators [p(X)|] and [p(T,)] are monotone because we as- 
sumed that X (or T) occurs only positively in g, and, except for negation, 
all other logical operators are monotone (the fixed-point operators as 
well, as we will see). Each monotone operator not only has unique least 
and greatest fixed-points, but these can be calculated iteratively, as stated 
in the following theorem. 


Remark 2.12. A formal definiton of ordinal numbers can be found in 
appendix A. For the moment, we think of them as a generalisation 
of the naturals numbers which allow to count beyond the finite. The 
first ordinal numbers are the natural numbers 0,1,2,... itself. The least 
infinite ordinal number is the set of all natural numbers, written as w, 
followed by w+1,w+2,...,w+2,w-2+ j es Nc 6 cere 
Definition 2.13. Let A be a set, and F : P(A*) + P(A*) be a monotone 
operator. We define the stages Xq of an inductive fixed-point process: 


Xo =D 
Xt = f (Xn) 
0. Ce U Xq for limit ordinals A. 
a<r 


Due to the monotonicity of F, the sequence of stages is increasing, i.e. 
Xq © Xg for a < B, and hence for some ¥, called the closure ordinal, 
we have X, = X41 = F(X,). This fixed-point is called the inductive 
fixed-point and denoted by Xo. 

Analogously, we can define the stages of a similar process: 


X°:= Ak 
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P Gaa— 0.4y) 
XA:= () x for limit ordinals A. 
a<A 


which yields a decreasing sequence of stages X* that leads to the induc- 
tive fixed-point X° := X7 for the smallest 7 such that XY = X7*1, 


Theorem 2.14 (Knaster, Tarski). Let F be a monotone operator. Then 
the least fixed-point lfp(F) and the greatest fixed-point gfp(F) of F 
exist, they are unique and correspond to the inductive fixed-points, i.e. 
lfp(F) = Xo, and gfp(F) = X®. 


To understand the inductive evaluation let us consider an example. 
We will evaluate the formula wX.(P V OX) on the following Kripke 
structure: 


K = ({0,...,n},{(i,i +1) |i <n}, {n}). 


The structure K represents a path of length n + 1 ending in a position 
marked by the predicate P. The evaluation of this least fixed-point 
formula starts with Xy = © and X; = P = {n}, and in stepi-+1 all 
nodes having a successor in X; are added. Therefore, X27 = {n —1,n}, 
X3 = {n—2,n —1,n}, and in general X, = {n-—k+1,...,n}. Finally, 
Xn41 = Xn+2 = {0,...,n}. As you can see, the formula pX.(P V OX) 
describes the set of nodes from which P is reachable. This example 
shows one motivation for the study of fixed-point logics: It is possible 


to express transitive closures of various relations in such logics. 


2.4 Model Checking Games for Fixed-Point Logics 


In this section we will see that parity games are the model checking 
games for LFP and L,. 

We will construct a parity game G(2, ¥(a@)) from a formula ¥(%) € 
LFP, a structure 2( and a tuple @ by extending the FO game with the 
moves 


fp Tx.p(T, x)](4) > (7,4) 
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and 
Tb + 9(T,b). 


We assign priorities O((a)) € IN to every instantiation of a subformula 
g(x). Therefore, we need to make some assumptions on ¥: 


e ¥ is given in negation normal form, ie. negations occur only in 
front of atoms. 
e Every fixed-point variable T is bound only once in a formula 
[fp Tx.p(T, x)]. 
¢ Ina formula [fp T%.¢~(T, x)] there are no other free variables besides 
X in @. 
Then we can assign the priorities using the following schema: 


¢ ()(T@) is even if T is a gfp-variable, and O(T@) is odd if T is an 
lfp-variable. 

e If T’ depends on T (i.e. T occurs freely in [fp T’x.p(T, T’, X)]), then 
Q(Ta) < O(T’D) for all a, b. 

¢ C(g(a@)) is maximal if p(@) is not of the form Ta. 


Remark 2.15. The minimal number of different priorities in the game 
G (21, ¥(a)) corresponds to the alternation depth of ¥. 


Before we provide the proof that parity games are in fact the appro- 
priate model checking games for LFP and L,,, we introduce the notion 
of an unfolding of a parity game. 

Let G = (V,Vo,V1,E,Q) be a parity game. We assume that the 
minimal priority in G is 0 and that all positions v € V with O(v) = 0 
have a unique successor, i.e., VE = {s(v) }. 

Let T= {v € V: O(v) = 0}. We define a modified game G™ = 
(V,Vo,Vi,E~,Q) with E~ = E\ (T x V),ie., positions in T are made 
terminal positions in G~. Further, we define a sequence of games G* that 
only differ from G~ in the assignment of the terminal positions in T to 
the players. For this purpose, we use a sequence of partitions (Tj, Ty) of 
T such that in G“, Player o wins at final positions v € T. The sequence 
of partitions is inductively defined depending on the winning regions 
W5 of the players in the games G* as follows: 


Cel Bees 
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¢ TAtT = {v © T: s(v) € We} for any ordinal a, 
© Th = Une, Th if A is a limit ordinal, 

e Ti = T \ Tj for any ordinal a. 

We have 

°WoD WED WED... D WED Wet. 
«WC Wr GWG ace WHC WET i. 


So there exists an ordinal « < |V| such that We = Wi*! = W® and 
= Sd) 
W* = wet] — we, 


Lemma 2.16 (Unfolding Lemma). 
Wo=Wo and W,=W>,. 


Proof. Let « be such that W5° = Wo and let f* be a positional winning 
strategy for Player 0 from W4 in G. Define: 


f"(v) ifvEV\T, 


yo VG Seen Oe 
s(v)  ifveE VoNT. 


A play 7t consistent with f that starts in W5° never leaves W5°: 


e If (i) € Wo \T, then z(i+1) = f*(7(i)) € Wh = WS (fa is a 
winning strategy in G*). 

° If (i) EWE AT = WENT = Wot1 NT, then r(i) € Ti*?, ie. (i) 
is a terminal position in G* from which Player 0 wins, so by the 
definition of T4*+! we have 7(i+ 1) =s(v) € Wt = We. 


Thus, we can conclude that Player 0 wins 71: 


e If 7 hits T only finitely often, then from some point onwards 71 is 
consistent with f*“ and stays in W§ which results in a winning play 
for Player 0. 

¢ Otherwise, 7t(i) € T for infinitely many i. Since we had Q(t) = 0 < 
O(v) for all v € V,t € T, the lowest priority seen infinitely often is 
0, so Player 0 wins 7t. 


For v € W?°, we define p(v) = min{B : 0 € wr} and let g? be 
a positional winning strategy for Player 1 on WwW, in GP. We define a 
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positional strategy g of Player 1 in G®™ by: 


gPO(v) ifoeWe\TAVy 
gia SV, « OF? < s(0) ifvETAV, 


arbitrary otherwise 


Let 7 = 7r(0)7r(1) ... be a play consistent with g and 7(0) € W;”. 
Claim 2.17. Let 7t(i) € W;°. Then 
(1) 2(i+1) € W®, 


(2) p(t + 1)) < p(7) 
(3) (i) € T > p(t +1)) < p((4)). 


(so mt(i) € wP). We have 


Proof. Case (1): (i) € WP? \ T, p(7(i)) = B 
i+1) € WP C W® and p(n(i+ 


m(i+1) = g(z(i)) = g?(m(i)), so m( 
1)) < B= pe(x()). 

Case (2): 7(i) € WP? NT, p(7(i)) = B. Then we have 7t(i) € W7°, B = 7+ 
1 for some ordinal 7, and 7(i + 1) = s(7(i)) € W/, so (i +1) © WP 
and p(7(i+1)) <7 < B=p(7(i)). Q.E.D. 


As there is no infinite descending chain of ordinals, there exists an 
ordinal B such that p(7(i)) = p(7(k)) = B for all i > k, which means 
that (i) ¢ T for alli > k. As 7t(k)71(k +1)... is consistent with ¢? and 
m(k) € wp, so 7t is won by Player 1. 

Therefore we have shown that Player 0 has a winning strategy 
from all vertices in W5° and Player 1 has a winning strategy from all 
vertices in W7?. As V = WS UW), this shows that Wo = W§° and 
W, = WP. Q.E.D. 


We can now give the proof that parity games are indeed appropriate 
model checking games for LFP and Ly. 


Theorem 2.18. If 21 | ‘Y(@), then Player 0 has a winning strategy in the 
game G(21,¥(a@)) starting at position ¥ (a). 


Proof. By structural induction over Y (a). We will only consider the inter- 
esting cases ¥(a) = |gfp Tx.p(T,x)|(a@) and ¥(a@) = [lfp Tx.p(T, x)](a). 
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Let ¥(@) = [gfp T%.¢~(T,X)|(@). In the game G(2, ¥(a)), the posi- 
tions Tb have priority 0. Every such position has a unique successor 
p(T,b), so the unfoldings G* (2, ¥(a)) are well defined. 

Let us take the chain of steps of the gfp-induction of g() on 2. 


NOS Di, ee Seer... 


We have 


i] 


t~ttvet 


Wt ¥(a) a € gfp(9”) 
a © X* for all ordinals a 
a © X**! for all ordinals a 


(21, X“) E g(a) for all ordinals a. 


Induction hypothesis: For every X Cc A‘ 


(2, X) — p(b) iff Player 0 has a winning strategy in 
G((2, X), p(a)) from pa). 


We show: If Player 0 has a winning strategy in G((2, X*), p(a@)) starting 
at position g(a), then Player 0 has a winning strategy in G* (2, ¥(@)) 
starting at position (a). 

By the unfolding lemma, the second statement is true for all ordinals 
a if and only if Player 0 has a winning strategy in G(2, ¥(a@)) starting at 
g(a). 

As (a) is the only successor of ¥(@) = [gfp TX.p(T, £)|(@), this 
holds exactly if Player 0 has a winning strategy in G(2l, ¥(a)) starting at 
Y (a). 

It remains to show that Player 0 has indeed a winning strategy in 
the game G( (2, X"), p(@)) starting at the position g(a). 

There are few differences between G((2, X*), p(@)) and the unfold- 
ing G* (2, ¥(a)): 


e In G* (2, ¥(a@)), there is an additional position ¥ (a), but this position 
is not reachable. 


¢ The assignment of the atomic propositions Tb: 
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— Player 0 wins at position Tb in G((2, X“), o(@)) if and only if 
be X*, 

— Player 0 wins at position Tb in G* (2, ¥(a)) if and only if Tb € 
Te. 


So we need to show using an induction over « that 
be X* iff Tb € Tf. 


Base case a = 0: X° = AK and T9 = T = {Tb: be A*}. 
Induction step « = y +1: Then b € X* = X**! if and only if (2, X7) - 
g(b), which in turn holds if Player 0 wins G((2, X7),@(b)) starting 
at g(b). By induction hypothesis, this holds if and only if Player 0 
wins the unfolding G7 (2, ¥(@)) starting at g(b) = s(Tb) if and only if 
Tei aie 
Induction step with « being a limit ordinal: We have that b € X* if b € X7 
for all ordinals y < a, which holds, by induction hypothesis, if and only 
if Tbe te for all 7 < a, which is equivalent to Tb € Tp. 

The proof for ¥ (a) = |lfp Tx.~(T, X)|(a@) is analogous. Q.E.D. 


2.5 Defining Winning Regions in Parity Games 


To conclude this chapter, we consider the converse question—whether 
winning regions in a parity game can be defined in fixed-point logic— 
and show that, given an appropriate representation of parity games as 
structures, winning regions are definable in the p-calculus. 

A parity game G = (V,VW,V,,E,Q) with priorities O(V) = 
{0,1,...,d — 1}, can be described by the Kripke structure Kg = 
(V,Vo,Vi,E,Po,...,Py_1) with atomic propositions P} = {v € V : 
O(0) = j}. 

Given the above representation, the p-calculus formula 

d—1 
py” = vXo.pX1.vXo....AXy-1 \V ((VoA PA OX )V 


j=0 
(Vi, AP; ADX;)), 


where A = v if d is odd, and A = p otherwise, defines the winning 
region of Player 0 in the sense of the following theorem. 
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Theorem 2.19. Kg,v | g™ if and only if Player 0 has a winning 


strategy from v in G. 


Proof. The model checking game for g" on Kg is essentially the same 
as the game G itself, up to the elimination of ‘stupid moves’: 


e Eliminate moves after which the opponent wins in at most two steps. 
For instance, Player 0 would never move to a position (Vo A P; A 
©Xj,0) if v was not a vertex of Player 0 or did not have priority j. 
Similarly, Player 1 would not move to a position (Pj,7) or (Vo,0) if 
VE P; or v € Vz. 

¢ Contract sequences of trivial moves and remove the intermediate 
positions. 

A schematic view of a model checking game for pg is sketched in 
Figure 2.2. Q.E.D. 
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Figure 2.2. Part of a model checking game for gy". 
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After our treatment of reachability (and safety) games in the first, and 
parity games in the second chapter, we now discuss infinite games in 
more general setting. More precisely, the games that we study are two- 
player, zero-sum games of perfect information, played on game graphs and 
admitting infinite plays. 

Formally, a graph game is a pair G = (G,Win) where G = 
(V, Vo, Vi, E,Q,) is a directed graph with V = VoU Vj and O: V > C for 
a set C of colours (or priorities) and a set Win C C“ of infinite sequences 
of colours. We call G the arena of G and Win the winning condition of G. 

As before a play of G is a finite or infinite sequence 71 = U90102... € 
V=“ such that (v;,0;41) € E for all i. A finite play is lost by the player 
who cannot move any more, and an infinite play 7t is won by Player 0 
if O(7t) = O(v9)O(v1)... € Win, otherwise Player 1 wins (there are no 
draws). Let Plays(G) denote the set of all plays of G and Ps,(G) be set 
of all initial segments x € V* of a play in Plays(G) 


3.1 Determinacy 


A strategy for Player 7 in a game G = (G, Win) is a function f : V*V, > 
V such that (v, f(xv)) € E for all x € V* and v € V,. Thus, a strategy 
maps prefixes of plays which end in a position in V, to legal moves of 
Player o. A play 7t = vpv,... is consistent with a strategy f for Player o if 
for all proper prefixes v9 ...0n of 7t such that uv, € Vo we have 0741 = 
f(vo..-Un). We say that f is a winning strategy from position vo if every 
play starting in vo that is consistent with f is won by Player o. The set 


Wo = {v € V: Player o has a winning strategy from v} 


is the winning region of Player 7. In zero-sum games it always holds that 
WoW, = ©. We call a game G determined if Wo UW, = V, ie. if from 
each position one player has a winning strategy. 
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We can generalize the notion of winning regions from initial posi- 
tions to arbitrary initial segments of plays. Let Wz be the set of those 
initial segments x € V* of plays for which Player ¢ has a strategy f to 
prolong x to a wining play (ie. every play of form x7 € Plays(G) that is 
consistent with f is won by Player 7). Clearly if Prin(G) = Wo U W, then 
G is determined. 

For determinacy questions it suffices to consider games played on 
trees and forests. Indeed, for an arena G with a node vg, let T(G, vo) 
be the tree obtained by unraveling G from vp. Obviously, a player has 
a winning strategy for (G, Win) from vp if, and only if she has one for 
(7 (G,v9),Win) for the root vp. For the forest F(G) := Useg 7 (G,2) 
we then have that (#(G),Win) is determined at the roots if, and only 
if, (G, Win) is determined. Notice further that, on trees and forests, all 
strategies are positional so in this case there is no difference between 
determinacy and determinacy via positional strategies. 

A classical and very old determinacy theorem is due to Zermelo 
who proved that a game of this kind is always determined if it only 
admits finite plays. A slightly stronger variant of this result, applying to 
games with infinite plays, is the following. 


Theorem 3.1 (Zermelo). Let G be a game such that in every play the 


winner is determined after finitely many moves. Then G is determined. 


Proof. The condition that the winner of every play is determined after 
finitely many moves means that every infinite play 7t of G has a finite 
initial segment x < 7 such that every play of form x7’ is won by the 
same player. We claim that this implies that Prin(G) = Wo UW, and 
hence the determinacy of G. 

Let X = Pfin(G) \ (Wo U W,), and assume, towards a contradiction, 
that X # ©. Take some x = yu € X, with v € V,. 

For all w € vE it follows that xw = yow ¢ Wo (because otherwise 
x € W,). Further, if we had that xw € W1_, for all w € vE, then also 
x € W,_,. Thus there exists some prolongation xw of x with xw € X. 

By induction, there exists an infinite play x7t such that xz € X 
for all finite prefixes z of 7. In particular the winner of x7t is not 
determined after any finite initial segment, which contradicts our initial 
assumption. Q.E.D. 
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The game that Zermelo originally wanted to study is Chess, which 
does not quite satisfy our definition of a game given above, since it ad- 
mits draws. One thus has to slightly modify the determinacy statement 
for Chess. 


Corollary 3.2. For Chess one of the following three possibilities holds: 


¢ White has a winning strategy. 
e Black has a winning strategy. 


¢ Both players have a strategy to enforce at least a draw. 


In the previous chapter, we proved a strong determinacy theorem 
for parity games. We now look for general properties of Win that guar- 
antee determinacy. To answer this question we shall need topological 
arguments. But before we develop them, we introduce the notion of a 
Gale-Stewart game and prove the existence of non-determined games. 


3.2 Gale-Stewart Games 


In this chapter we will show that, using the Axiom of Choice, one 
can construct a non-determined game. Later, we will mention which 
topological properties guarantee determinacy and how this is related to 
logic. 

Let B be an alphabet (for instance B = {0,1} or B = w). Ina 
Gale-Stewart game the players alternately choose symbols from B in 
an infinite sequence of moves and thus create an infinite word 7t € BY. 
Gale-Stewart games can be described as graph games in different ways. 
For B = {0,1}, for example, as a game on the infinite binary tree 


ae = ({0, 1 Vo, Vi, Ey O); 
where 
Vee AOA", 
new 
Vea UO 
new 


E = {(x,xi) : x € {0,1}*,i € {0,1}}, 
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and: {0,1}* > {0,1Le}: eH ¢,xi H i. Alternatively, it can be 
described as a game on the graph depicted in Figure 3.1. Similar game 
graphs can be defined for arbitrary B. 


(0) fo 


ws 


Figure 3.1. Game graph for Gale-Stewart game over B = {0,1} 


1 


Theorem 3.3 (Gale-Stewart). There exists a non-determined Gale-Stewart 


game. 


We shall present two proofs. The first one uses enumerations of the 
strategy spaces of the two player via ordinals (see Appendix A) up to 
2”. The second uses ultrafilters. Both rely on the Axiom of Choice (AC). 


Proof. For any countable alphabet B with at least two symbols, let To = 
{x € B* : |x| even} and T; = {x € B* : |x| odd}. Then 


F={f:T) > B} andG= {g:T; > B} 


are the sets of strategies for Player 0 and for Player 1. Since B is countable, 
|F| = |G| = |P(w)| = 2”. Thus, using the well-ordering principle 
(which is equivalent to AC) we can enumerate the strategies by ordinals 
up to 2°: 


BPeaVfei ee 2" and G = (ene tea 20 


For strategies f and g let f*g € B® be the uniquely determined play 
arising from f and g. We shall construct two increasing sequences of 
sets X,,Y, C BY for a < 2” such that 

(1) Xa N Yn = @, 
(2) |Xa|,|Ya] < 2%, 

Let Xp = Yo = ©. For a successor ordinal a = 6 + 1 consider the 

strategy fg. The cardinality of Xg and Y¢ is smaller than 2“ but there 
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are 2” different strategies g € G and thus 2” different plays that are 
consistent with fg. Hence there exists one that is not in Xg. Choose such 
a play fg°g (AC again) and add it to Yg to construct Y, : Yg U {fps}. 
Analogously, choose a play f*gg that is consistent with gg and which 
is not in Yz, and construct Xy := XgU {f°gg}. For limit ordinals A let 
Xq = Upea Xp and Yq := Usgea Yep. 

We claim that the Gale-Stewart game with winning condition Win := 
Ugeow Xq is not determined. 

Indeed, assume that f = fx, for some a < 2”, is a winning strategy 
for Player 0. By the construction of Win, there is a strategy g € G such 
that fx°g € Yy41 and thus fy°g ¢ Win, a contradiction. 

Now assume that g = @y, for some a < 2”, is a winning strategy for 
Player 1. Analogously, there is a strategy f € F such that f° gy € Xa41 © 
Win, a contradiction as well. Q.E.D. 


The second proof that we shall present uses the concept of an 
ultrafilter. We first recall the definition of a filter. 


Definition 3.4. Let I be a non-empty set. A non-empty set F C P(1) isa 
filter if 


(1) Q€F, 
Qxer, yer => any € F, and 
QaeCE fox = Yet. 


The intuition behind a filter is that it is a family of large sets. 
Example 3.5. The set {x C w: w \ x is finite} is a filter. We call it the 
Fréchet filter. 

Definition 3.6. An ultrafilter is a filter that satisfies the additional re- 
quirement: 

(4) for all x C I either x € ForI\x € F. 
Example 3.7. Fix n € w. Then U, = {a Cw: n € a} is an ultrafilter. 
Ultrafilters of this form are called principal ultrafilters. 


Every ultrafilter U that contains a finite set must be principal. Oth- 
erwise U would contain a smallest set a which is not a singleton. Pick 
some n € a. Since {n} ¢ U, the complement w \ {n} is in U, and hence 
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also its intersection with a. But aM (w \ {n}) =a\ {n} € a contradicting 
the minimality of a in U. 

On the other side, an ultrafilter that does not contain a finite set 
must contain all co-finite ones, and thus extend the Fréchet filter. But 
the Fréchet filter is not an ultrafilter and it is not obvious that it can be 
extended to one in a consistent way. The proof that this is possible uses 
Zorn’s Lemma or the Compactness Theorem for propositional logic. It 
holds for every set F C P(w) such that a4, ---Nam 4 @ for all m € N, 
Ay,...,Am € F. 


Theorem 3.8. The Fréchet filter F can be expanded to an ultrafilter 
USF 


Proof. Let F be the Fréchet filter. We use propositional variables X, for 
every a € P(w). Let ® = By U Pp where 


by = {7Xo} 
U {X, A Xp > Xgryp 14,0 Cw} 
U {X, > X,:aCb,a,b Cw} 
UA eI Say, 
and 
®r = {X,:a€ F}. 


Every model Z of ® defines an ultrafilter U which expands F, 
namely U = {a C w: I(X,) = 1}. It remains to show that ® is 
satisfiable. 

By the compactness theorem, it suffices to show that every finite 
subset of ©@ is satisfiable. Hence, let Bp be a finite subset of ©. Then the 
set Fy = {a € F: X, € Po} is also finite. Now consider the following 
two cases: 


e Fy = ©. Define the interpretation Z by 


I(X) = 1 if0€a, 
. 0 otherwise. 


Then Z — ®p. 
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e Fy = {a1,...,am}. Since F is a filter, there exists np € 4, +--+ am. 
Define the interpretation Z by 


1(X,) = 1 ifno€a 
: 0 otherwise 


Again, we have ZT — ®p. 


Hence, ®o is satisfiable. Q.E.D. 


We are now able to give an alternative construction for non- 
determined games. Let U be an ultrafilter that expands the Fréchet 
filter. We construct a Gale-Stewart game over B = w with winning 
condition Winy as follows. Player 0 wins a play x = xox1... € w if 


e Player 1 has played a number that is not higher than the previously 
played one, ie. min{j : xj41 < xj} exists and is even, or 
ex <x <%<... and 


A(x) == [0,x0) U LU [xai41, x2i42) € U 


i€w 


Figure 3.2. The winning condition of the ultrafilter game 


Proposition 3.9. The Gale-Stewart game with winning condition Winy 


is not determined. 


Proof. Towards a contradiction, assume that Player 0 has a winning 
strategy f. We construct two plays x and x’, both of which are consistent 
with f. 

e In the first play the opening move x9 = f(¢) of Player 0 is answered 
by Player 1 with an arbitrary number x; > xo. The second move of 
Player 0 is then x = f(xox1). 

¢ In the second play x’, Player 1 uses x2 as her answer to the opening 
move x9 = f(€) by Player 0. The second move of Player 0 in the play 
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x’ is then x3 = f (xox2), and Player 1 uses this in the play x as her 
answer to X9X1X2. 

¢ This is iterated. In play x, Player 1 extends in her (i+ 1)st move the 
sequence X9x1...X2; by X2j41 = f(xXox2x3...X2;), Le. she just copies 
the (i + 1)st move of Player 0 in play x’. 

¢ Similarly, in play x’, Player 1 answers the initial segment 
XoXox3... X41 by X2i42 = f(xX0xX1...X2i+1), ie she copies the i + 1st 
move of Player 1 in x. 


Thus, in both plays, Player 1 essentially uses the strategy f itself as a 
counterstrategy against f. 


0 Xo x2 x4 
ee 2 ae f re 

1 : : 7 = *5 
; a 


Figure 3.3. Playing the Ultrafilter game 


This results in two plays x = xox x2... and x! = xpXx2x3x4..., where 
Xoi40 = f(xXox1..-X2i41) but also x94, = f(x0x1...%2;). Both plays 
are consistent with the winning strategy f for Player 0. Thus we have 
A(x) € Uand A(x’) € U. But 


A(x) = [0,x0) UL [x2i+1,x2:42) 
i€w 
and 
A(x’) = [0,x0) U L) [x2i+2,2i43). 
iew 
Thus A(x) A(x’) = [0,x9) € U. However, since U expands the Fréchet 
filter, the co-finite set w \ [0,x9) is in U and thus [0,x9) ¢ U, a contradic- 
tion. 
Analogously, one derives a contradiction from the assumption that 
Player 1 has a winning strategy. Q.E.D. 
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3.3 Topology 


Definition 3.10. A topology on a set S is defined by a collection of open 
subsets of S. It is required that 


¢ @, and S are open; 

e if X and Y are open, then X 1 Y is open; 

e if {X;:i¢€ I} isa family of open sets, then Uj<) X; is open. 
If O C P(S) is a collection of open sets, we call the pair (S,O) a 
topological space. 


Often, a topology is defined by its base: A set B of open subsets of S 
such that every open set can be represented as a union of sets in B. 


Example 3.11. The standard topology on R is defined by the base consist- 
ing of all open intervals (a,b) C R. 

In our setting, we will only be concerned with the following topol- 
ogy on B“”, which is due to Cantor. Its base consists of all sets of the form 
zt :=z- B® for z € B*. Consequently, a set X C B” is open if it is the 
union of sets z+, i.e. if there exists a set W C B* such that X = W- BY”. 
Moreover, a set X C B” is closed if its complement B” \ X is open. For 
B = {0,1}, this topology is called the Cantor space, and for B = w it is 
called the Baire space. 


WwW 
ral B 


Figure 3.4. Base sets in the Cantor space 


Example 3.12. 


¢ The base sets z+ are both open and closed (clopen) since we have 
BY \ zt = W,- BY where W, = {y € B* | y & zandz < y}. (Here, 
u <v means that u is a prefix of v.) 

e 0*1{0,1}° is open. The complement {0} is closed, but not open. 
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° Ly = {x € w” : x contains d infinitely often} = () (w*-d)"-w® is 


new 
a countable intersection of open sets. 
Next, we will give another useful characterisation of closed sets. A 
tree T C B* is a prefix-closed set of finite words, ie., z © T and y < z 
implies y € T. For a tree T let [T] be the set of infinite paths through T 
(note: T C B*, but [T] C BY). 


Example 3.13. Let T = 0* = {0": n € w}. Then [T] = {0°}. 


Lemma 3.14. X C B“ is closed if and only if there exists a tree T C B* 
such that X = [T]. 


Proof. 
(=>) Let X be closed. Then there is a W C B* such that BY \ X = W- BY. 
Let T:= {w € B* | Vz(z < w > z € W)}. T is closed under prefixes 
and [T] = X. 

(<=) Let X = [T]. For every x ¢ [T] there exists a smallest prefix 
Wy <x such that w, ¢ T. Let W := {wy : x € X}. Then BY \ X = W- BY 
is open, thus X is closed. Q.E.D. 


We call a set W C B® prefix-free if there is no pair x,y € W such that 
ey, 


Lemma 3.15. 


(1) For every open set A C B” there is a prefix-free set W C B* such 
that A = W- BY’. 

(2) Let B be a finite alphabet. A C B“ is clopen if and only if there is a 
finite set W C B* such that A = W- BY”. 


Proof. For (1), let A = U- B” for some open U C B*. Define 
W := {w € U: U contains no proper prefix of w}. 


W is prefix-free and W- BY’ = U- BY = A. 

For (2) let A C B” be clopen. Thus there exist prefix-free sets 
U,V C B* such that A = U- BY and BY’ \ A = V- BY. We will show 
that U UV is finite. Let T = {w © B* | w has no prefix in UU V}. If T 
is finite, then U U V is also finite: if U (or V) would be infinite, then T 
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would be infinite as well, since it contains all proper prefixes of elements 
of U (respectively V). Hence it suffices to show that T is finite. Notice 
that T is a finitely branching tree (since B is finite) that contains no infinite 
path, since otherwise there exists an infinite word x € B“ corresponding 
to this path with x € U- BY UV- BY = AU (B® \ A) = B”. By K@énig’s 
Lemma, this implies that T is finite. 

For the converse, let A = W- BY” where W C B* is finite. Let 
1 = max{|w| : w € W}. Then BY \ A = Z- BY where 


Z = {z € B®: |z| =1 and no prefix of z is in W}. 
Thus, A is clopen. Q.E.D. 
Notice that (2) does not hold for infinite alphabets B. 


Definition 3.16. Let T = (S,O) be a topological space. The class of Borel 
sets is the smallest class B C P(S) that contains all open sets and is 
closed under countable unions and complementation: 


ee Oe oi 
elf X € BthenS\ X € B; 
elf {X,:n €w} CB then Unew Xn € B. 


Most of the w-languages L C B” occurring in Computer Science 
are Borel sets. Borel sets form a natural hierarchy of sets £2 and I1° for 
0 < a < w, where w is the first uncountable ordinal number. 

© 112 =coX? := {S\ X: X € x9} for every a; 
eS Mis Ky oe eI for B < x} for a > 0. 
We are especially interested in the first levels of the Borel hierarchy: 
° ae Open sets 
¢ II): Closed sets 
¢ ©3: Countable unions of closed sets 
¢ IIS: Countable intersections of open sets 
¢ X29: Countable unions of IT$-sets 


e mie Countable intersections of x9-sets 
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Example 3.17. Let d € B. 
La = {x € B® : x contains d infinitely often} = () (B*-d)"- BY. 
a mas 
Hence, Ly € ITS. ex} 
To determine the membership of an w-language in a class X° or 
I1° of the Borel hierarchy and to relate the classes, we need a notion of 
reducibility between w-languages. 


Definition 3.18. A function f : B’ > C® is called continuous if f—!(Y) 
is open for every open set Y C C”. 

Let X C BY, Y C C”. We say that X is Wadge reducible to Y, X < Y, 
if there exists a continuous function f : B’ + C® such that f~!(Y) = X, 
ie. x € X iff f(x) € Y for all x € BY. For any such function f, we write 
ie. eo 


Exercise 3.1. Prove that the relation < satisfies the following properties: 


eX <YandY < Z imply X < Z; 
e X < Y implies BY \ X < C’\ Y. 


Theorem 3.19. Let X < Y for Y € X° (or Y € TH®). Then X € XO 
(respectively X € IT°). 


Proof. The claim is true by definition for X9 (the open sets) and thus also 
for IT?. 
For a > 1, let f : X < Yand Y € 22. We have that Y = Unew Yn 


where Yn € Ugea IT}. Define X;, := f~!(Yn). Then X, < Yj; for all 


n € w,and thus, by induction hypothesis, Xn € Ug<a Tp. We have: 


xEX & f(x) EY 
= f(x) € Yn for some n € w 


& x €X, forsomen € w. 
Hence, X = Unew Xn € LP. Q.E.D. 
In the following we will present a game-theoretic characterisation 


of the relation < in terms of the so-called Wadge game. 
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Definition 3.20. Let X C BY’, Y C C”. The Wadge game W(X,Y) is an 
infinite game between two players 0 and 1 who move in alternation. In 
the i-th round, Player 0 chooses a symbol x; € B, and afterwards Player 1 
chooses a (possibly empty) word y; € C*. After w rounds, Player 0 has 
produced an w-word x = x9x1X2--- € BY, and Player 1 has produced 
a finite or infinite word y = yoy1y2--- € C=”. Player 1 wins the play 
(x,y) if, and only if, ye C’ andxeE xX oye y. 


Example 3.21. Let B = C = {0,1}. 


e Player 1 wins W(0*1{0,1}”, (0*1)%). 
Winning strategy for Player 1: Choose 0 until Player 0 chooses 1 for 
the first time. Afterwards, always choose 1. 

¢ Player 0 wins W((0*1)”,0*1{0,1}”). 
Winning strategy for Player 0: Choose 1 until Player 1 chooses a 
word containing 1 for the first time. Afterwards, always choose 0. 


Theorem 3.22 (Wadge). Let X C BY”, Y C C”. Then X < Y if and only if 
Player 1 has a winning strategy for W(X, Y). 


Proof. 

(<=) A winning strategy of Player 1 for W(X,Y) induces a mapping 
f : BY — C® such that x € X iff y € Y. It remains to show that f is 
continuous. Let Z = U- C“” be open. For every u € U denote by V;,, the 
set of all words v = x9x1...Xn € B* such that u is the answer of Player 1 
tov, ie. u = f(x) f (x1)... f (xn). Then f-'(U-C”) = V- BY where 
V := Uneu Vis 

(=) Let f : X < Y. We construct a strategy for Player 1 as fol- 
lows. Player 1 has to answer Player 0’s moves x9x1x2... by an w-word 
Yo¥iY2..., but Player 1 can delay choosing y; until he knows x9x1...Xn 
for some appropriate n > i. 

Choice of yo: Consider the partition BY = Weecf '(c-C”). Since 
c-C” is clopen, f~!(c-C“) is also clopen. For every x € B“” there exists 
c € C such that x € f-!(c-C”), and since f~!(c -C”) is clopen, there 
is a prefix w, < x such that w,-BY C f-'(c-C”). So Player 1 can 
wait until Player 0 has chosen a prefix w € B* that determines the set 
f-'(c-C”) the word x will belong to and choose yo = c. 
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The subsequent choices are done analogously. Let yo... yj; € C* be 
Player 1’s answer to xo9...X, € B*. For the choice of yj; we consider 
the partition 


Xo++ +X, BY = ).Lat Yet yp ce OC"): 


Since the sets f-!(yo---y;-c-C“”) are clopen, after finitely many moves, 
by choosing a prolongation x9 ° +» XnXy41 °° + X;, Player 0 has determined 
in which set f~!(yo-+-y;:c-C®”) the word x will be. Player 1 then 
chooses yj41 = Cc. 

By using this strategy, Player 1 constructs the answer y = f(x) for 
the sequence x chosen by Player 0. Otherwise, there would be a smallest 
i such that y; 4 f(x;). This is impossible since x € f~'(yo---y;-C®). 
Since f : X < Y, we have x € X iff y EY. Q.E.D. 


Definition 3.23. A set Y C C” is £°-complete if: 


“Yer: 
eX <Y forall Xe xX?. 


I1°-completeness is defined analogously. 
Note that Y is £2-complete if, and only if, C’ \ Y is IT?-complete. 


Proposition 3.24. Let B = {0,1}. Then: 
* 0*1{0,1}” is X?-complete; 
° {0} is II?-complete; 
° {0,1}*0” is X$-complete; 
° (0*1)” is I19-complete. 


Proof. By the above remark, it suffices to show that 0*1{0,1}@ and (0*1)” 
are X!-complete and I1$-complete, respectively. 
¢ We know that 0*1{0,1}“” € X?. Let X = W- BY be open. We 
describe a winning strategy for Player 1 in W(X,0*1{0,1}”): Pick 0 
until Player 0 has completed a word contained in W; from this point 
onwards, pick 1. Hence, X < 0*1{0,1}. 
* We know that (0*1)% € TI9. Let X = Myew Wn: BY € TIS. We 
describe a winning strategy for Player 1 in W(X, {0,1}*0%): Start 
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with 7 := 0; for arbitrary i, answer with 1 and set i := 1+ 1 if the 
sequence x9... x, of symbols chosen by Player 0 so far has a prefix 
in W;, otherwise answer with 0 and leave i unaffected. Q.E.D. 


3.4 Determined Games 


We call a game G = (V, Vo, Vi, E, Win) clopen, open, closed, etc., or sim- 
ply a Borel game, if the winning condition Win C V“ has the respective 
property. 

Clopen games are basically finite games: If A C B” is clopen, then 
for every x € B“ there exists a finite prefix wy < x such that: 


elIfxe Athen w,t C A; 
eIfx ¢ A then wxt C BY \ A. 


Thus, by Zermelo’s Theorem, clopen games are determined. 
A stronger result is the following: 


Theorem 3.25. Every open game, and thus every closed game, is deter- 
mined. 


Proof. Let G = (V, Vo, Vi, E, Win) where Win = U - V“ is open. First, we 
consider finite plays: Let T, = {v € Vj_, : vE = @} and Ag = Attro(Tz). 
From every position v € A, Player o wins after finitely many moves 
with the attractor strategy. 

For the infinite plays consider 


Gg’ = G{V\ (AgUA}) 


with positions V’ := V \ (Aj U A}). In G’ every play is infinite, and 
Player 0 wins 7 = v9v102... if and only if 7 € U-V“. Obviously, 
Player 0 wins in G/ starting from vo if she can enforce a sequence 
V9 ...Un € U. Then every infinite prolongation of this sequence is a 
play inU-V”. 

Instead of G’ we consider again the equivalent game on the trees 
T(v) = Tg(v), the unfolding of G from v € V. Positions in 7(v) are 
words over V: T(v) C V*. Now consider the set 


Bo = {v EV’: ve Attr, (U-V*)} 
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of positions from where player 0 can enforce a play prefix in U- V*. 
From every position in V’ \ Ao, Player 1 has a strategy to guarantee 
that the play never reaches U - V* since V’ \ Ag is a trap for Player 0. 
But a play that never reaches U - V* is won by Player 1. It follows that 
Wo = Ao U Bo and W; = A; U(V’ \ Bo), and thus V = Wo UW. QED. 


A much stronger result was established by Donald Martin in 1975. Its 
proof is beyond the scope of these lecture notes. 


Theorem 3.26 (Martin). All Borel games are determined. 


Here are some winning conditions for frequently used games in 
Computer Science: 


e Muller conditions: Let B be finite, 77 C P(B), F; = P(B) \ Fo. 
Player 0 wins zt € B“ if and only if 


Inf(7t) := {b € B: b appears infinitely often in 71} € F,. 
Hence, the winning condition is the set 


{PEB? tint) Se Fo) = Ch) Lal Be \aea)), 


XEFo dex d€éX 


a finite Boolean combination of IT3-sets. 


¢ Parity conditions (see previous chapter) are special cases of Muller 
conditions and thus also finite Boolean combinations of I13-sets. 


° Every w-regular language is a Boolean combination of IT3-sets. 
This follows from the recognisability of w-regular languages by 
Muller automata and the fact that Muller conditions are Boolean 
combinations of IT$-sets. 


In practice, winning conditions are often specified in a suitable logic: 
w-words x € B® are interpreted as structures 2, = (w,<,(P,)peR) with 
unary predicates P, = {i € w: x; = b}. A sentence (for example 
in FO, MSO, etc.) over the signature {<}U{P, : b € B} defines the 
w-language (winning condition) L(w) = {x € BY : Ay — yp}. 
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Example 3.27. Let B = {0,...,m}. The parity condition is specified by 
the FO sentence 


p:= f\ (ave (y<z— Pz) Vv A Vyaz(y <zA re)) ; 


c<b 


We have: 


¢ FO and LTL define the same w-languages (winning conditions); 
¢ MSO defines exactly the w-regular languages; 
e There are w-languages that are definable in MSO but not in FO; 


¢ w-regular languages are Boolean combinations of IT9-sets. 


In particular, graph games with winning conditions specified in LTL, 
FO, MSO, etc. are Borel games and therefore determined. 


3.5 Muller Games and Game Reductions 


Muller games are infinite games played over an arena G = (Vo, V1,E,QO: 
V > C) with a winning condition depending only on the set of priorities 
seen infinitely often in a play. It is specified by a partition P(C) = 
Fo U Fi, and a play 7 = v9v1v2... is won by Player o if 


Inf(7t) = {c : O(v;) =c for infinitely many i € w} € F,. 


We will only consider the case that the set C of priorities is finite. Then 
Muller games are Borel games specified by the FO sentence 


VV (A Vxdy(x <yAPy)A [\ Axvy(x<y “re 


XEF, \cEx céX 


So Muller games are determined. Parity conditions are special Muller 
conditions, and we have seen that games with parity winning conditions 
are even positionally determined. The question arises what kind of 
strategies are needed to win Muller games. Unfortunately, there are 
simple Muller games that are not positionally determined, even solitaire 
games. 
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Example 3.28. Consider the game arena depicted in Figure 3.5 with the 
winning condition Fp = {{1,2,3}}, ie. all positions have to be visited 
infinitely often. Obviously, player 0 has a winning strategy, but no 
positional one: Any positional strategy of player 0 will either visit only 
positions 1 and 2 or positions 2 and 3. 


Ome O=eO 


Figure 3.5. A solitaire Muller game 


Although Muller games are, in general, not positionally determined, 
we will see that Muller games are determined via winning strategies 
that can be implemented using finite memory. To this end, we introduce 
the notions of a memory structure and of a memory strategy. Although 
we will not require that the memory is finite, we will use finite memory 
in most cases. 


Definition 3.29. A memory structure for a game G with positions in V 
is a triple SN = (M, update, init), where M is a set of memory states, 
update : M x V + Misa memory update function and init: V + Misa 
memory initialisation function. The size of the memory is the cardinality of 
the set M. 

A strategy with memory IN for Player o is given by a next-move 
function F : Vz x M - V such that F(v,m) € vE for all v € Vo,m € M. 
If a play, from starting position vg, has gone through positions v9? .. . On, 
the memory state is (v9... 0), defined inductively by m(vo) = init(vo), 
and m(vg...0;Vj41) = update(m(v9 ...0;),0;41), and in case vy, € V, the 
strategy leads to position F(vn,m(v9...,0n)). 


Remark 3.30. In case |M| = 1, the strategy is positional, and it can be 
described by a function F : Vz > V. 


Definition 3.31. A game G is determined via memory 2M if it is deter- 
mined and both players have winning strategies with memory Jt on 
their winning regions. 


Example 3.32. In the game from Example 3.28, Player 0 wins with a 
strategy with memory It = ({1,3}, update, init) where 
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¢ init(1) = init(2) = 1, init(3) =3 and 
Ib OG a leo) 
° update(m,v) = Se 
m ifv=2. 
The corresponding strategy is defined by 


2 ifve {1,3}, 
F(v,m)= <3 ifv=2,m=1, 
I ioe] 2m =. 


Let us consider a more interesting example now. 


Example 3.33. Consider the game DJW) with its arena depicted in Fig- 
ure 3.6. Player 0 wins a play 7 if the maximal number in Inf(7z) is equal 
to the number of letters in Inf(7t). Formally: 


Fo = {X C {1,2,4,b} : |X {a, b}| = max(XN {1,2})}. 


(6 2 


Figure 3.6. Muller game G = DJW2 


Player 0 has a winning strategy from every position, but no po- 
sitional one. Assume that f : {a,b} — {1,2} is a positional winning 
strategy for Player 0. If f(a) = 2 (or f(b) = 2), then Player 1 always 
picks a (respectively b) and wins, since this generates a play 77 with 
Inf(7t) = {a,2} (respectively Inf(7r) = {b,2}). If f(a) = f(b) = 1, then 
Player 1 alternates between a and b and wins, since this generates a play 
7m with Inf(7t) = {a,b,1}. However, Player 0 has a winning strategy that 
uses the memory depicted in Figure 3.7. The corresponding strategy is 
defined as follows: 


eH) 1 “if m= c#d, 
Cin) = 
2 if m= #cd. 


63 


3 Infinite Games 


b,1,2 


Figure 3.7. Memory for Player 0 


Why is this strategy winning? If from some point onwards Player 1 
picks only a or only b, then, from this point onwards, the memory state 
is always b#a or at#b, respectively, and according to F Player 0 always 
picks 1 and wins. In the other case, Player 1 picks a and b again and 
again and the memory state is #ab or #ba infinitely often. Thus Player 0 
picks 2 infinitely often and wins as well. 


The memory structure used in this example is a special case of the 
LAR memory structure, which we will use for arbitrary Muller games. 
But first, let us look at a Muller game with infinitely many priorities that 
allows no winning strategy with finite memory but one with a simple 


infinite memory structure: 


Example 3.34. Consider the game with its arena depicted in Figure 3.8 
and with winning condition Fy = {{0}}. It is easy to see that every 
finite-memory strategy of Player 0 (the player who moves at position 0) is 
losing. A winning strategy with infinite memory is given by the memory 
structure IN = (w, init, update) where init(v) = v and update(m,v) = 
max(m,v) together with the strategy F defined by F(0,m) =m +1. 


Given a game graph G = (V,Vo,V;,E) and a memory structure 
IM = (M, update, init), we obtain a new game graph 


Gx M= (V x M, Vo x M, Vy x M, Eupdate) 
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AN 


Figure 3.8. A game where finite-memory strategies do not suffice 


n 


where 
Eupdate = {((v,1), (v',m')) : (v,0') € E and m’ = update(m, 0’) }. 


Obviously, every play (vo, mo)(v1,11)... in G x It has a unique 
projection to the play vov;... in G. Conversely, every play vo, v1,... in 
G has a unique extension to a play (vo, mo) (V1,11)... in G x MN with 
mo = init(v9). 

Definition 3.35. For games G = (G,O, Win) and G’ = (G’,’, Win’), we 
say that G reduces to G’ via memory IN, G <m G’, if G’ = G x IN and 
every play in G’ is won by the same player as the projected play in G. 


Given a memory structure 9% for G and a memory structure 9’ for 
G x MN, we obtain a memory structure N* = M x IM’ for G. The set of 


memory locations is M x M’, and we have memory initialisation 
init*(v) = (init(v), init’ (v, init(v))) 
with the update function 


update*((m,m'),v) = 
(update(m,v), update’ (m’, (v, update(m,v)))). 
Theorem 3.36. Suppose that G reduces to G’ via memory 9 and that 
Player o has a winning strategy for G’ with memory 90’ from position 


(vp, init(vp))). Then Player 7 has a winning strategy for G with memory 
M x Mt’ from position vo. 


Proof. Given a strategy F’ : (V, x M) x M’ > (V x M) for Player o 
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in G’, we have to construct a strategy F : (Vs x (M x M’)) > V for 
Player g in G. For any v € V, and any pair (m,m') © M x M’ we 
have that F’((v,m),m') = (w,update(m,w)) for some w € vE. We put 
F(v,(m,m')) = w. If a play in G that is consistent with F proceeds from 
position v with current memory location (m,m’) to a new position w, 
then the memory is updated to (n,n’) with n = update(m,w) and n! = 
update’ (m’, (w,n)). In the extended play in G’, we have an associated 
move from (v,m) to (w,n) with memory update from m’ to n'. Thus, 
every play in G from initial position vo that is consistent with F is the 
projection of a play in G’ from (vo, init(vo)) that is consistent with F’. 
Therefore, if F’ is a winning strategy from (vo, init(vo)), then F is a 
winning strategy from v9. Q.E.D. 


Corollary 3.37. Every game that reduces via memory SM to a positionally 


determined game is determined via memory 90. 


Obviously, memory reductions between games can be composed. 
If G reduces to G’ with memory IN = (M, update, init) and G’ reduces 
to G” with memory St’ = (M’, update’, init’) then G reduces to G” with 
memory (M x M’, update”, init”) where 

init” (v) = (init(v), init’ (v, init(v))) 
and 

update” ((m,m'),v) = 

(update(m,v), update’ (m’, (v, update(m,v)))). 


The classical example of a game reduction with finite memory is 
the reduction of Muller games to parity games via latest appearance 
records. Intuitively, a latest appearance record (LAR) is a list of priorities 
ordered by their latest occurrence. More formally, for a finite set C of 
priorities, LAR(C) is the set of sequences cj ... cy#cx41...c; of elements 
from C U {#} in which each priority c € C occurs at most once and 
# occurs precisely once. At a position v, the LAR cy... cy#cx41...C] is 
updated by moving the priority Q(v) to the end, and moving # to the 
previous position of QO(v) in the sequence. For instance, at a position 
with priority cz, the LAR cyc2c3#c4cs5 is updated to cy#c3c4c5C2. (If O(0) 
did not occur in the LAR, we simply append Q(v) at the end). Thus, 
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the LAR memory for an arena with priority labelling O: V — C is the 
triple (LAR(C), update, init) with init(v) = #0O(v) and 


update(c,...cy#cp41...C],0) = 


C1... Cgdtepyy...cO(v) if O(v) € {c1,...c7}, 


Civ p tease: OO) = ce 


The hit set of an LAR c,...cg#cp41...c; is the set {cp41...c;} of 
priorities occurring after the symbol #. Note that if in a play 77 = vgv, ... 
the LAR at position vy is cy... cg#ex44.-- cz, then O(0,) = c; and the hit 
set {cyi1...c;} is the set of priorities that have been visited since the 
latest previous occurrence of c; in the play. 


Lemma 3.38. Let 7t be a play of a Muller game G with finitely many 
priorities, and let Inf(zz) be the set of priorities occurring infinitely often 
in 7t. Then the hit set of the latest appearance record is, from some point 
onwards, always a subset of Inf(7) and infinitely often coincides with 
Inf(7). 


Proof. For each play 77 = v9v1v2... there is a position v,, such that 
O(vn) € Inf(7t) for all n > m. Since no priority outside Inf(7z) is 
seen after position Vm, the hit set will, from that position onwards, al- 
ways be contained in Inf(7t), and the LAR will always have the form 
Cy... Cj—-1Cj.-. Ck#Cgy1--.C; Where cj,...cj-1 remains fixed and 
Inf(7t) = {c;,---,cy}. Since all priorities in Inf(7r) are seen again and 
again, it happens infinitely often that, among these, the one occurring 
leftmost in the LAR is hit. At such positions, the LAR is updated to 
C1,..+,Cj-1#Cj41--.c)C;, and the hit set coincides with Inf(7r). Q.E.D. 


Theorem 3.39. Every Muller game with finitely many priorities reduces 
via LAR memory to a parity game. 


Proof. Let G be a Muller game with game graph G, priority labelling 
OQ: V + C and winning condition (Fo, 1). We have to prove that 
G <:ar GQ’ for a parity game G’ with game graph G x LAR(C) and an 
appropriate priority labelling QO’ on V x LAR(C), which is defined as 
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follows: 


2k if {Opa : sgt € Fo, 


Orci cee see) = 
2k+1 = if Lepigpeses ey} CF i: 


Let 77 = v9v1V2... be a play on G and fix a number m such that, 
for all n > m, O(v,) € Inf(7t) and the LAR at position v,, has the form 
Cy... CjCj41 ++. CeCe -..cy Where Inf(7r) = {cj41,...c;} and the prefix 
cy...¢; remains fixed. In the corresponding play 7t' = (v9,10)(01,11) --- 
in G’, all nodes (0n,1) for n > m have a priority 2k + p with k > j and 
o € {0,1}. Assume that the play 7 is won by Player ¢, i.e., Inf(7t) € Fo. 
Since the hit set of the LAR coincides with Inf(7z) infinitely often, the 
minimal priority seen infinitely often on the extended play is 2] + ¢. 
Thus the extended play in the parity game G’ is won by the same player 
as the original play in G. Q.E.D. 


Corollary 3.40. Muller games are determined via finite memory strate- 
gies. The size of the memory is bounded by (|C| +1)!. 


The question arises which Muller conditions (Fo, 71) guarantee 
positional winning strategies for arbitrary game graphs? One obvious 
answer are parity conditions. But there are others: 


Example 3.41. Let C = {0,1}, Fp = {C} and Fy; = P(C)\{C} = 
{{0}, {1},@}. (Fo, 71) is not a parity condition, but every Muller game 
with winning condition (Fo, F1) is positionally determined. 


Definition 3.42. The Zielonka tree for a Muller condition (Fo, F;) over 
C is a tree Z(Fo, F,) whose nodes are labelled with pairs (X,a) such 
that X € F,. We define Z(Fo, 71) inductively as follows. Let C € F, 
and Co,...,C,_1 be the maximal sets in {X C C: X € F,_,}. Then 
Z(Fo, F1) consists of a root, labelled with (C, 7), to which we attach as 
subtrees the Zielonka trees Z(F9 MN P(C;), 719 P(C;)), i =0,...,k —1. 


Example 3.43. Let C = {0,1,2,3,4} and Fp = {{0,1}, {2,3,4}, {2,3}, 


{2,4}, {3}, {4}}, Fy, = P(C) \ Fo. The Zielonka tree Z(Fo, F,) is de- 
picted in Figure 3.9. 
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Fa | 
{0,1},0 {2,3,4},0 
a y 


{0},1 {1},1 fot {3,4},1 


{3},0 {4},0 


O,1 O,1 


Figure 3.9. A Zielonka tree 


A set Y C C belongs to F, if there is a node of Z(Fo, F,) that is 
labelled with (X,) for some X D Y and for all children (Z,1— 0) of 
(X,7) we have Y ¢ Z. 


Example 3.44. Consider again the tree Z(Fo, 71) from Example 3.43. It is 
the case that {2,3} € Fo, since ({2,3,4},0) is a node of Z(Fo, F,) and 


0 {2,3} 12,3415 
© {2,3} Z (2h; 
e{2,3)¢ {3,4}. 
The Zielonka tree of a parity-condition is especially simple, as 
Figure 3.10 shows. 
Besides parity games there are other important special cases of 
Muller games. Of special relevance are games with Rabin and Streett 
conditions because these admit positional winning strategies for one 


player. 


Definition 3.45. A Streett-Rabin condition is a Muller condition 
(Fo, F1) such that Fo is closed under union. 


In the Zielonka tree for a Streett-Rabin condition, the nodes labelled 
with (X,1) have only one successor. It follows that if both Fy and F; are 
closed under union, then the Zielonka tree Z(Fo, F;) is a path, which 
implies that (Fo, 71) is equivalent to a parity condition. 
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{m—2,m—1},o 


{m—1},l-¢0 


Figure 3.10. The Zielonka tree of a parity-condition with m priorities 


In a Streett-Rabin game, Player 1 has a positional winning strategy 
on his winning region. On the other hand, Player 0 can win on his 
winning region via a finite-memory strategy, and the size of the memory 
can be directly read off from the Zielonka tree. We present an elementary 
proof of this result. 


Theorem 3.46. Let G = (V, Vo, Vi, E,O.) be a game with a Streett-Rabin 
winning condition (Fo,;). Then G is determined, ie. V = WoU 
W,, with a finite memory winning strategy for Player 0 on Wo, and a 
positional winning strategy for Player 1 on Wj. The size of the memory 
required by the winning strategy for Player 0 is bounded by the number 
of leaves of the Zielonka tree Z(Fo, F1). 


Proof. We proceed by induction on the number of priorities in C or, 
equivalently, the depth of the Zielonka tree Z(Fo,F;). Let / be the 
number of leaves of Z(Fo, F;). We distinguish two cases. 

Case 1: C € F,. Let 


x Player 0 has a winning strategy with memory 
[= 40: , 
e of size <1 from v 


and X; = V \ Xo. It suffices to prove that Player 1 has a positional 
winning strategy on X;. To construct this strategy, we combine three 
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positional strategies of Player 1: A trap strategy, an attractor strategy, 
and a winning strategy on a subgame with fewer priorities. 

At first, we observe that Xj is a trap for Player 0. This means that 
Player 1 has a positional trap strategy t on X; to enforce that the play 
stays within Xj. 

Since Fg is closed under union, there is a unique maximal subset 
C’ C Cwith C’ € Fp. Let Y:= X,NQ71(C\ C’), and let Z = Attr,(Y) \ 
Y. Observe that Player 1 has a positional attractor strategy a, by which 
he can force, from any position z € Z, that the play reaches Y. 

Finally, let V’ = X \ (YUZ) and let G’ be the subgame of G induced 
by V’, with winning condition (Fp) P(C’), FP (C’)) (see Figure 3.11). 
Since this game has fewer priorities, the induction hypothesis applies, 
i.e. we have V’ = Wj U Wj, and Player 0 has a winning strategy with 
memory < / on Wj, whereas Player 1 has a positional winning strategy ¢’ 
on W;. However, Wj = ©: Otherwise we could combine the strategies of 
Player 0 to obtain a winning strategy with memory < / on Xj) UW§ 2 Xo, 
a contradiction to the definition of Xo. Hence W; = V’. 


Xo 


vy Z., Y X1 


(ome (ae O-1(C\ Cc’) 


Figure 3.11. Constructing a winning strategy for Player 1 
We can now define a positional strategy g for Player 1 on X; by 


g(x) ifxeV’, 
g(x) =<a(x) ifxeZ, 
f(x) ifxey. 


Consider any play 7t that starts at a position v € X, and is consistent 
with g. We have to show that zt is won by Player 1. Obviously, 7t stays 
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within X,. If it hits Y U Z only finitely often, then from some point 
onwards it stays within V’ and coincides with a play consistent with 
g’. It is therefore won by Player 1. Otherwise, 7 hits Y U Z, and hence 
also Y, infinitely often. Thus, Inf(7t) N (C \ C’) 4 @ and Inf(7r) € Fy. So 
Player 1 has a positional winning strategy on X. 

Case 2: C € Fo. There exist maximal subsets Co,...,Cr_4 © C with 
C; € F, (see Figure 3.12). Observe that if D(C \ C;) 4 © for alli << k 
then D € Fo. Now let 


X, := {v € V: Player 1 has a positional winning strategy from v}, 


and Xo = V \ X;. We claim that Player 0 has a finite memory winning 
strategy of size < / on Xo. To construct this strategy, we proceed in a 
similar way as above, for each of the sets C \ C;. We will obtain strategies 
fo,---,fk—-1 for Player 0 such that each fj has finite memory Mj;, and 
we will use these strategies to build a winning strategy f on Xo with 
memory Mp U---U Mg-_1. 


(C,0) 
(Co, 1) (Cred) 2S (Cy_1,1) 


Figure 3.12. The top of the Zielonka tree Z(Fo, F;) 


Fori=0,...,k—1, let ¥; = Xp»NOQ71(C\ C;), and Z; = Attro(Y;) \ 
Y;, and let a; be a positional attractor strategy by which Player 0 can 
force a play from any position in Z; to reach Y;. Furthermore, let 
U; = Xo \ (Y¥; U Z;), and let G; be the subgame of G induced by U; with 
winning condition (Fo 1 P(Ci), 719 P(C;)). The winning region of 
Player 1 in G; is empty: Indeed, if Player 1 could win G; from v, then, 
by the induction hypothesis, he could win with a positional winning 
strategy. By combining this strategy with the positional winning strategy 
of Player 1 on Xj, this would imply that v € Xj, but v € U; C V \ Xj. 

Hence, by the induction hypothesis, Player 0 has a winning strategy 
f; with finite memory M; on Uj. Let (f; + 4;) be the combination of f; 
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with the attractor strategy a;, defined by 


(fit a)(o) = ff) UO 
aj(v) ifv € Z;. 

From any position v € U; U Z; this strategy ensures that the play either 
remains inside U; and is winning for Player 0, or that it eventually 
reaches a position in Yj. 

We now combine the strategies (fo + 40),.-.,(fe_-1 + 4-1) to a win- 
ning strategy f on Xo, which ensures that either the play ultimately 
remains within one of the regions Uj; and coincides with a play ac- 
cording to fj, or that it cycles infinitely often through all the regions 
Vopang Lea 

At positions in Y := (<x Yj, Player 0 just plays with a (positional) 
trap strategy t ensuring that the play remains in Xo. At the first position 
v0 € Y, Player 0 takes the minimal i such that v ¢ Yj, ie. v € Uj; U Zi, 
and uses the strategy (f; + a;) until a position w € Y; is reached. At this 
point, Player 0 switches from i to j = i+ m (mod k) for the minimal m 
such that w ¢ Y;. Hence w € Uj U Z;; Player 0 now plays with strategy 
(f; + 4;) until a position in Y; is reached. There Player 0 again switches 
to the appropriate next strategy, as he does every time he reaches Y. 

Assuming that M; Mj; = © for i # j, it is not difficult to see that f 
can be implemented with memory M = Mo U---U Mg_1. We leave the 
formal definition of f as an exercise. 

Note that, by the induction hypothesis, the size of the memory M; 
is bounded by the number of leaves of the Zielonka subtrees Z(F9M 
P(Ci), F719 P(C;)). Consequently, the size of M is bounded by the 
number of leaves of Z(Fo, 1). 


It remains to prove that f is winning on Xo. Let 7 be a play that 
starts in Xo and is consistent with f. If 77 eventually remains inside 
some Uj, then from some point onwards it coincides with a play that is 
consistent with f; and is therefore won by Player 0. Otherwise, it is easy 
to see that 7c hits each of the sets Yo,..., Y,_1 infinitely often. But this 
means that Inf(7z) 1 (C \ C;) 4 @ for all i < k; as observed above this 
implies that Inf(7r) € Fo. Q.E.D. 
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An immediate consequence of Theorem 3.46 is that parity games 
are positionally determined. 


3.6 Complexity 


We will now determine the complexity of computing the winning regions 
for games over finite game graphs. The associated decision problem is 


Given: Game graph G, winning condition (Fo, ;) in a suitable 
encoding, v € V,a € {0,1}. 
Question: v € W,? 


For parity games, we already know that this problem is in NP MN coNP, 
and it is conjectured to be in P. Moreover, for many special cases, we 
know that it is indeed in P. Now we will examine the complexity of 
Streett-Rabin games and games with arbitrary Muller conditions. 


Theorem 3.47. Deciding whether Player 7 wins from a given position in 
a Streett-Rabin game is 


¢ coNP-hard for o = 0, 
e NP-hard for 7 = 1. 


Proof. It is sufficient to prove the claim for 7 = 1 since Streett-Rabin 
games are determined. We will reduce the satisfiability problem for 
Boolean formulae in CNF to the given problem. For every formula 


Ve NG. Gra NG 
i j 


in CNF, we define the game Gy as follows: Positions for Player 0 are the 
literals X1,...,X,, 7X1,..., 7X; occurring in ¥; positions for Player 1 
are the clauses Cj,...,C,. Player 1 can move from a clause C to a literal 
Y € C; Player 0 can move from Y to any clause. The winning condition 
is given by 


Fo ={Z:{X,-=X} C Z for at least one variable X}. 


Obviously, (Fo, 1) is a Streett-Rabin condition. 
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We claim that Y is satisfiable if and only if Player 1 wins Gy (from 
any initial position). 

(=) Assume that ¥ is satisfiable. There exists a satisfying interpre- 
tation Z : {X1,...,X,} — {0,1}. Player 1 moves from a clause C toa 
literal Y € C such that [Y]*? = 1. In the resulting play only literals with 
[Y]~ = 1 are seen, and thus Player 1 wins. 

(<=) Assume that Y is unsatisfiable. It is sufficient to show that 
Player 1 has no positional winning strategy. Every positional strategy f 
for Player 1 chooses a literal Y = f(C) € C for every clause C. Since ¥ is 
unsatisfiable, there exist clauses C,C’ and a variable X such that f(C) = 
X, f(C’) = 3X. Otherwise, f would define a satisfying interpretation 
for Y. Player 0’s winning strategy is to move from —X to C and from 
any other literal to C’. Then X and —X are seen infinitely often, and 
Player 0 wins. Thus, f is not a winning strategy for Player 1. If Player 1 
has no positional winning strategy, he has no winning strategy at all. 

Is ¥ ++ Gy a polynomial reduction? The problem that arises is 
the winning condition: Both Fp and 7; contain exponentially many 
sets. Moreover, the Zielonka tree Z(Fo, F;) has exponential size. On 
the other hand, Fo and F; can be represented in a very compact way 
using a Boolean formula in the following sense: Let (Fo, 71) be a Muller 
condition over C. A Boolean formula Y with variables in C defines the 
set Fy = {Y CC: Zy — ¥} where 


Tyo) 1 ifceY 
yle) = 
0 ifc¢ Y. 


Y defines (Fo, F;) if Fy = Fo (and thus F_y = F;). Representing 
the winning condition by a Boolean formula makes the reduction a 
polynomial reduction. Q.E.D. 


Another way of defining Streett-Rabin games is by a collection of 
pairs (L, R) with L,R CC. The collection {(L1,R1),..., (Lx, Rx) } defines 
the Muller condition (Fo, 7) given by: 


Ppa 6 Cee eo Ss NINR SO forall < kh, 
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We have: 


e Every Muller condition defined by a collection of pairs is a Streett- 
Rabin condition. 

e Every Streett-Rabin condition is definable by a collection of pairs. 

¢ Representing a Streett-Rabin condition by a collection of pairs can 
be exponentially more succinct than a representation by its Zielonka 
tree or an explicit enumeration of Fo or #,: There are Streett- 
Rabin conditions definable with k pairs such that the corresponding 
Zielonka tree has k! leaves. 


The reduction ¥ ++ Gy can be modified such that the winning 
condition is given by 2m pairs, where m is the number of variables in YT: 


Lo = {Xi}, Roa ={7Xi}, Lar = {Xi}, Raj-1 = {Xi}. 


For the Streett-Rabin condition defined by {(L1,R1),...,(Lam,Ram)} we 
have that 


Z contains a Literal X; (or —X;) such that the 
F, = 4 Z: complementary literal =X; (respectively X;) is 


not contained in Z 


The winning strategies used in the proof remain winning for the modi- 
fied winning condition. 
To prove the upper bounds for the complexity of Streett-Rabin 


games we will consider solitaire games first. 


Theorem 3.48. Let G be a Streett-Rabin game such that only Player 0 
can do non-trivial moves. Then the winning regions Wo and W, can be 
computed in polynomial time. 


Proof. Let us assume that the winning condition is given by the collection 
P = {(L1,R}),..., (Le, Re) } of pairs. It is sufficient to prove the claim 
for Wo since Streett-Rabin games are determined. Every play 7 will 
ultimately stay in a strongly connected set U C V such that all positions 
in U are seen infinitely often. Therefore, we call a strongly connected set 
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U good for Player 0 if for alli <k 
OU) ALG FOSS OU) Ri. 


For every such U, Attro(U) C Wo. If U is not good for Player 0 then 
there is a node in U which violates a pair (Lj, R;). In this case Player 0 
wants to find a (strongly connected) subset of U where she can win 
nevertheless. We can eliminate the pairs (L;, R;) where O(U) NL; = © 
since they never violate the winning condition. On the other hand, 
Player 0 loses if a node of 


U = {u € U| A(u) € L; for some i such that Q(U) NR; = } 
is visited again and again. Thus we will reduce the game from U to U \ U 


with the modified winning condition P’ = {(L;,R;) € P: Q(U)NL; A 
@}. This yields Algorithm 3.1. 


Algorithm 3.1. A polynomial time algorithm solving solitaire Streett- 
Rabin games 


Algorithm WinReg(G, P) 
Input: Streett-Rabin game with game graph G and pairs condition P. 
Output: Wo, the winning region for Player 0. 


Wo = QO; 
Decompose G into its SCCs; 
For every SCC U do 
Pl = {(Li, Ri): O(U) NL; FO}; 
U := {u € U: O(u) € L; for some i such that O(U) VR; = ©}; 
if U = @ then W := WUU; 
else W := WU WinReg(G lintuy Ps 
enddo; 
Wo = Attro(W); 
Output Wo; 


The SCC decomposition can be computed in linear time. The de- 
composition algorithm will be called less than |V| times, the rest are 
elementary steps. Therefore, the algorithm runs in polynomial time. 
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It remains to show that Wo = WinReg(G, P): 

(C) Let v € Wo. Player 0 can reach from v a strongly connected set 
S that satisfies the winning condition. S is a subset of an SCC U of G. If 
U satisfies the winning condition, then v € WinReg(G, P’). Otherwise, 
S C U\U, and S is contained in an SCC of G ie The repetition of 
the argument leads to S C W and therefore v € WinReg(G, P) 

(>) Let v € WinReg(G,P). The algorithm finds a strongly con- 
nected set U (an SCC of a subgraph) that is reachable from v and that 
satisfies the winning condition. By moving from v into U, staying there, 
and visiting all positions in U infinitely often, Player 0 wins. Thus 
v € Wo. Q.E.D. 


Theorem 3.49. Deciding whether Player 7 wins from a given position in 
a Streett-Rabin game is 


¢ coNP-complete for 7 = 0, 
¢ NP-complete for 7 = 1. 


Proof. It suffices to prove the claim for Player 1 since Wo is the comple- 
ment of W,. Hardness follows from Theorem 3.47. To decide whether 
v € Wi, guess a positional strategy for Player 1 and construct the in- 
duced solitaire game, in which only Player 0 has non-trivial moves. 
Then decide in polynomial time whether v is in the winning region of 
Player 1 in the solitaire game (according to Theorem 3.48), i.e. whether 
the strategy is winning from v. If this is the case, accept; otherwise 
reject. Q.E.D. 


Remark 3.50. The complexity of computing the winning regions in arbi- 
trary Muller games depends to a great amount on the representation of 
the winning condition. For any reasonable representation, the problem 
is in Pspace, and many representations are so succinct as to render the 
problem Psrace-hard. Only recently, it was shown that, given an explicit 
representation of the winning condition, the problem of deciding the 


winner is in P. 
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4Basic Concepts of Mathematical 
Game Theory 


Up to now we considered finite or infinite games 


with two players, 


played on finite or infinite graphs, 


with perfect information (the players know the whole game, the 
history of the play and the actual position), 


with qualitative (win or loss) winning conditions (zero-sum games), 


with w-regular winning conditions (or Borel winning conditions) 


specified in a suitable logic or by automata, and 


with asynchronous interaction (turn-based games). 


Those games are used for verification or to evaluate logic formulae. 

In this section we move to concurrent multi-player games in which 
players get real-valued payoffs. The games will still have perfect infor- 
mation and additionally throughout this chapter we assume that the set 
of possible plays is finite, so there exist only finitely many strategies for 
each of the players. 


4.1 Games in Strategic Form 


Definition 4.1. A game in strategic form is described by a tuple T = 
(N, (Si)ien, (Pi)icn) where 

e N = {1,...,n} is a finite set of players 

e S; is a set of strategies for Player i 

e pi: S + Ris a payoff function for Player i 
and S := S; x --- x S, is the set of strategy profiles. T is called a zero-sum 


game if icy pi(s) = 0 for alls € S. 


The number p;(s1,...,Sn) is called the value or utility of the strategy 
profile (s1,...,S) for Player i. The intuition for zero-sum games is that 
the game is a closed system. 
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Many important notions can best be explained by two-player games, 
but are defined for arbitrary multi-player games. 

In the sequel, we will use the following notation: Let T be a game. 
Then S_; := S, x +--+ x Sj_1 X Sj41 X +++ X Sy is the set of all strategy 


profiles for the players except i. For s € S; and s_; € S_j, (s,s_j) is 
the strategy profile where Player i chooses the strategy s and the other 
players choose s_j. 


Definition 4.2. Let s,s’ € S;. Then s dominates s' if 


¢ for all s_; € S_; we have p;(s,s_;) > p;(s’,s_;), and 
there exists s_; € S_; such that p;(s,s_;) > pj(s’,s_j). 


A strategy s is dominant if it dominates every other strategy of the player. 


Definition 4.3. An equilibrium in dominant strategies is a strategy profile 
(s1,...,Sn) € S such that all s; are dominant strategies. 


Definition 4.4. A strategy s € 5S; is a best response to s_; € S_j; if 
pi(s,s_;) > pi(s’,s_;) for all s’ € Sj. 

Obviously, a dominant strategy is a best response to all strategy 
profiles of the other players. 

Example 4.5. The Prisoner’s Dilemma. 

Two suspects are arrested, but there is insufficient evidence for a 
conviction. Both prisoners are questioned separately, and are offered 
the same deal: if one testifies for the prosecution against the other and 
the other remains silent, the betrayer goes free and the silent accomplice 
receives the full 10-year sentence. If both stay silent, both prisoners are 
sentenced to only one year in jail for a minor charge. If both betray 
each other, each receives a five-year sentence. So this dilemma poses the 
question: How should the prisoners act? 


stay silent betray 
stay silent (—1,—1) (—10,0) 
betray (0,—10) (-—5,-—5) 


An entry (a,b) at position i, j of the matrix means that if profile (i, /) is 
chosen, Player 1 (who chooses the rows) receives payoff a and Player 2 
(who chooses the columns) receives payoff b. 


80 


4.2 Nash equilibria 


Betraying is a dominant strategy for every player, call this strategy 
b. Therefore, (b,b) is an equilibrium in dominant strategies. Problem: 
The payoff (—5, —5) of the dominant equilibrium is not optimal. 

The Prisoner’s Dilemma is an important metaphor for many deci- 
sion situations, and there exists extensive literature concerned with the 
problem. Especially interesting is the situation, where the Prisoner’s 
Dilemma is played repeatedly, possibly infinitely often. 


Example 4.6. Battle of the sexes. 


meat fish 
red wine (2,1) (0,0) 
white wine (0,0) (1,2) 


There are no dominant strategies, and thus there is no dominant equi- 
librium. The pairs (red wine, meat) and (white wine, fish) are distin- 
guished since every player plays with a best response against the strategy 
of the other player: No player would change his or her strategy unilater- 
ally. 


4.2 Nash equilibria 


Definition 4.7. A strategy profile s = (s1,...,5,) € S isa Nash equilibrium 
in T if for all i € N and all strategies si € S; 


pilSivS4) > ils’ 624): 
Sy 


S 


Thus, in a Nash equilibrium, every player plays with a best response 
to the profile of his opponents, and thus has no incentive to deviate 
unilaterally to a different strategy. Is there a Nash equilibrium in every 
game? The following example shows that this is not always the case, at 
least not in pure strategies. 


Example 4.8. Rock, paper, scissors. 
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rock scissors paper 

rock (0,0) (1,-1) (-—1,1) 

scissors (—1,1) (0,0) (1,-—1) 
paper (1,-1) (-1,1) (0,0) 


There are no dominant strategies and no Nash equilibria: For every pair 
(f, 2) of strategies one of the players can change to a better strategy. 
Note that this game is a zero-sum game. 

Although there are no Nash equilibria in pure strategies in rock, 
paper, scissors, there is of course an obvious good method to play this 
game: Randomly pick one of the three actions with equal probability. 
This observation leads us to the notion of mixed strategies, where the 
players are allowed to randomise over strategies. 


Definition 4.9. A mixed strategy of Player i in T is a probability distribu- 
tion py; : S; + [0,1] on S; (so that ses, u(s) = 1). 

A(S;) denotes the set of probability distributions on S;. A(S) := 
A(S1) x --- x A(S,) is the set of all strategy profiles in mixed strate- 
gies. 

The expected payoff is p;: A(S) > R, 


Pian cha) = (Tots) misr--90 


(S1,-5n)ES \JEN 


For every game [ = (N, (Sj)ien, (pi)ien) we define the mixed expansion 
T = (N, (A(Si))ien, (Bien): 

Definition 4.10. A Nash equilibrium of T in mixed strategies is a Nash 
equilibrium in I, ie. a Nash equilibrium in IT’ in mixed strategies is 
a mixed strategy profile 7 = (p1,...,fn) € A(S) such that, for every 
player i and every 1! € A(S), pili ti) > Pill): 

Nash equilibria (in mixed strategies) provide the arguably most 
important solution concept in classical game theory (although, as we 
shall point out later, this concept is not without problems). An important 
reason for the success of Nash equilibrium as a solution concept is the 
fact that every finite game has one. To prove this, we shall use a well- 
known classical fixed-point theorem. 
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Theorem 4.11 (Brouwer’s Fixed-Point Theorem). Let X C IR” be compact 
(i.e., closed and bounded) and convex. Then every continuous function 
f:X — X has a fixed point. 


We do not prove this here but remark that, interestingly, the Brouwer 
Fixed-Point Theorem can itself be proved via a game-theoretic result, 
namely the determinacy of HEX. 


Theorem 4.12 (Nash). Every finite game I in strategic form has at least 
one Nash equilibrium in mixed strategies. 


Proof. Let T = (N,(Si)ien, (pi)ien). Every mixed strategy of Player i 
is a tuple yw; = (pis)ses, € [0,1]|$! such that Lses, His = 1. Thus, 
A(S;) © [0,1]!5i is a compact and convex set, and the same applies to 
A(S) = A(S;) x +--+ x A(S,) for N = {1,...,n}. For every i € N, every 
pure strategy s € S; and every mixed strategy profile 7 € A(S) let 


8is(H) = max (pj(s, 7_;) — pi(F),0) 


be the gain of Player i if he unilaterally changes from the mixed profile 
# to the pure strategy s (only if this is reasonable). 

Note that if 9;;(#) = 0 for all i and alls € S;, then 7 is a Nash 
equilibrium. We define the function 


f: A(S) > A(S) 
v > f (7) = (V1,...,Un) 
where v; : S; > [0,1] is a mixed strategy defined by 


His + 8is(F) 


1+ Vises; 8i,s (7) 


Vis = 


For every Player i and alls € S;, # ++ v;, is continuous since pj; is 
continuous and thus gj, too. f(f#) = (V1,-..,Vn) is in A(S): Every 
Vj = (Vis)ses, is in A(S;) since 


3 = Lises; Mis + Uses; 8is(B) = ire Seas 8is(B) = 
séS; es 1 a ses: 8is(B) 1+ Uses; Sis (71) 
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By the Brouwer fixed point theorem f has a fixed point. Thus, there is a 
# € A(S) such that 


is = His + 8is(2) 
ee Uses; 8i,s (7) 


for alli and all s. 

Case 1: There is a Player i such that Vises, 8i,s(#) > 0. 

Multiplying both sides of the fraction above by the denominator, we get 
His * Uses; 8is(H) = 8is(#)- This implies pj, = 0 << gis(#) = 0, and 
thus g;5(7) > 0 for all s € S; where p1;, > 0. 

But this leads to a contradiction: g;;(7/) > 0 means that it is prof- 
itable for Player i to switch from (p/;, u_;) to (s, #_;). This cannot be true 
for all s where p/;, > 0 since the payoff for (4;,4_;) is the mean of the 
payoffs (s, 4_;) with arbitrary pu; ;. However, the mean cannot be smaller 
than all components: 


Pi Mia) =), fees PSH) 


seES; 


a ye His * Pils, Hi) 
seS; 
His>0 


> )o Mis: PilHir Hi) 
seS; 
His>0 


= pi(Hir Ui) 


which is a contradiction. 
Case 2: gi 5(@) = 0 for alli and all s € S;, but this already means that 77 
is a Nash equilibrium as stated before. Q.E.D. 


The support of a mixed strategy pi; € A(S;) is supp(p;) = {s € S;: 
Hi(s) > O}- 


Theorem 4.13. Let y* = (1,...,fn) be a Nash equilibrium in mixed 
strategies of a game I’. Then for every Player i and every pure strategy 


s,s’ € supp(p;) 


Pil, U2) = Pils ea): 
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4.3 Two-person zero-sum games 


Proof. Assume pj;(s,#-i) > pi(s’,¥—i). Then Player i could achieve a 
higher payoff against p_; if she played s instead of s’: Define ji; € A(S;) 
as follows: 


> Ppi(Hir fi) 


which contradicts the fact that p is a Nash equilibrium. Q.E.D. 


4.3 ‘Two-person zero-sum games 


We want to apply Nash’s Theorem to two-person games. First, we note 
that in every game I = ({0,1}, (So, $1), (po, P1)) 


max min ,g)< min = max @). 


The maximal payoff which one player can enforce cannot exceed the 
minimal payoff the other player has to cede. This is a special case of the 
general observation that for every functionh: X x Y— R 


sup inf h(x,y) < inf suph(x,y). 
x sy y x 


(For all x',y: h(x’,y) < sup,h(x,y). Thus inf, h(x’,y) < inf, sup, 
h(x,y) and sup, inf, h(x,y) < inf, sup, h(x,y).) 


Remark 4.14. Another well-known special case from mathematical logic 


is that dx Vy Rxy — Vy dx Rxy. 


Theorem 4.15 (v. Neumann, Morgenstern). 
Let T = ({0,1}, (So,51), (p, —p)) be a two-person zero-sum game. For 
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every Nash equilibrium (f*, 9*) in mixed strategies 


max min => = min max 
main pfs) =P(f 8) = eee ie) 


In particular, all Nash equilibria have the same payoff which is called 
the value of the game. Furthermore, both players have optimal strategies 
to realise this value. 


Proof. Since (f*,¢*) is a Nash equilibrium, for all f € A(So), g € A(S1) 


RE APT BIS PGs ): 


Thus 


oe pfs) =e, (fe) =e pfs"). 


So 


max min > > min max 
ek a ) PF, 8) Dee) aoe p(f,8) 


and 


max min ,2)< min max 
pee nt PE 8) geA(S;) fEd(So) PF -8) 


imply the claim. Q.E.D. 


4.4 Regret minimization 


To motivate the concept of regret minimization we consider 
Example 4.16. Traveller’s Dilemma. This is a symmetric two-player game 


ie ({1,2}, (Si, So), (pi, P2)) with S1 = So = {2,. F ., 100} and 


BaP A: IAS Y; 


pilwy) = \y—-2 ify<x, p2(x,y) = prly,x) 
x fx 4, 
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The only Nash equilibrium in pure strategies is (2,2) since for each 
(i,j) with i # j the player that has chosen the greater number, say i, 
can do better by switching to j — 1, and also, for every (i,i) with i > 2 
each player can do better by playing i — 1 (and getting the payoff i+ 1 
then). Also most other solution concepts from game theory (such as 
the iterated elimination of dominated strategies discussed in the next 
section) suggest that the players should choose 2. 


However, experiments show that people (even game theorists!) tend 
to select large numbers, in the range between 90 and 100; moreover they 
seem right to do so, since they perform much better in these experiments 
than those who follow what game theory proposes and select the strategy 
a 


The question arises whether there are alternative solution concepts 
that justify the choice of large strategies in the Traveller’s Dilemma, and 
if yes, which one. A relatively recent proposal that seems to achieve 
this is regret minimization. When a player uses this concept, he wants to 
minimize the lost payoff (which he would “regret”) due to not playing 
with the best response to the strategies of the other players. 


This idea was formulated in the context of decision theory, con- 
cerned with the choices of individual agents rather than the interaction 
of different agents as in game theory. Accordingly, the payoff is de- 
termined by a binary function p : S x Z — R, where S is the set of 
strategies of the player we are considering, and Z is an abstract set of 
possible states. 

Before we can introduce regret minimization, we need several defi- 
nitions. In state z € Z, the maximal payoff for our player is 


p’(z) := max p(s,Z), 


seS 


and if the player chooses the strategy s € S, he will miss the following 
payoff: 


regret ,(s,z) := p"(z) — p(s,Z). 


The overall maximal regret for the strategy s is 
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maxreg_,(s) := maxreegret,(s,Zz). 
g,(s) := max regret, (s,z) 


Now, the decision with respect to regret minimization would be: Choose 
s € S such that maxreg,(s) is minimal. 

Let us reconsider Example 4.16. Since it belongs to game theory, Z 
is the set of strategy profiles of the other players. We claim that exactly 
the strategies s € {96,...,100} minimize the maximal regret. To see this, 
note that for those s, we have that maxreg,(s) = 3, since 


e if t <s, then p(s,t) > f—2 and p*(t) < t+ 1, thus regret,(s,t) = 
p*(t) — p(s,t) <#+1—(¢—2) =3, 
e iff > s, then p(s,t) =s+2 and p*(t) < 101, thus regrety(s,t) < 
101:-= @422) 99s <3, 
and on the other hand, 


° regret,(96, 100) = 101 — 98 = 3, 
e fors € {97,...,100}, regrety(s,96) = 97 — 94 = 3. 


Also, for s < 95, we have that maxregy(s) > 4, as maxreg,(s) > 
regrety(s, 100) = 101 — (s +2) =99—s>4. 

Consequently, regret minimization suggests a strategy s with 96 < 
s < 100. We will now iterate this idea. If both players eliminate strategies 
which do not minimize the regret, we obtain a subgame with strategies 
{96,...,100}. In this game, we have that 


SS” 


° maxreg,(97) = 2, since 


— regret,(97, 100) = 101 — 99 = 2, 
— regret,(97,99) = 100 —99 = 1, 
— regret, (97,98) = 99 —99 = 0, 

— regrety(97,97) = 98 —97 = 1, 

— regrety (97,96) = 96 — 95 = 2. 


e maxreg,(100) > regret,(100,99) = 100 — 97 = 3. 
maxregp(99) > regretp(99,98) = 99 — 96 = 3. 

° maxreg,(98) > regret, (98,97) = 98 — 95 = 3. 
maxreg,(96) > regret,(96,100) = 101 — 98 = 3. 


Hence, 97 is the unique strategy which minimizes the regret in this sub- 
game and thus is the choice of a player who assumes that his opponent 
wants to minimize his regret as well. 
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4.5 Iterated Elimination of Dominated Strategies 


Besides Nash equilibria and (iterated) regret minimization, the iterated 
elimination of dominated strategies is a promising solution concept for 
strategic games which is inspired by the following ideas. Assuming 
that each player behaves rational in the sense that he will not play a 
strategy that is dominated by another one, dominated strategies may 
be eliminated. Assuming further that it is common knowledge among 
the players that each player behaves rational, and thus discards some of 
her strategies, such elimination steps may be iterated as it is possible 
that some other strategies become dominated due to the elimination 
of previously dominated strategies. Iterating these elimination steps 
eventually yields a fixed point where no strategies are dominated. 


Example 4.17. 


L R i R 

T (1,0,1)(1,1,0)]| (1,0,1) (0,1,0) 

B (1,1,1) (0,0,1) | (1,1,1) (1,0,0) 
be Y 


Player 1 picks rows, Player 2 picks columns, and Player 3 picks matrices. 


¢ No row dominates the other (for Player 1); 
¢ no column dominates the other (for Player 2); 
¢ matrix X dominates matrix Y (for Player 3). 


Thus, matrix Y is eliminated. 


e In the remaining game, the upper row dominates the lower one (for 
Player 1). 


Thus, the lower row is eliminated. 
¢ Of the remaining two possibilities, Player 2 picks the better one. 
The only remaining profile is (T, R, X). 


There are different variants of strategy elimination that have to be 
considered: 


¢ dominance by pure or mixed strategies; 
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© (weak) dominance or strict dominance; 


¢ dominance by strategies in the Jocal subgame or by strategies in the 
global game. 


The possible combinations of these parameters give rise to eight different 
operators for strategy elimination that will be defined more formally in 
the following. 

Let T = (N,(S;)icn, (pi)icn) such that S; is finite for every Player i. 
A subgame is defined by T = (Tj,...,Tn) with T; C S; for all i. Let 
ui € A(S;), and s; € S;. We define two notions of dominance: 


(1) Dominance with respect to T: 
Wi >T 8; if and only if 
© pi(mi,t_i) = pi(si,t_;) for all t_; € T_; 
© pi(ui,t_i) > pi(si,t_;) for some t_; € T_j. 
(2) Strict dominance with respect to T: 
Ui >t s; if and only if p;(pi,t_;) > pi(s;,t_;) for all t_; € T_j;. 


We obtain the following operators on T = (Tj,...,Tn), T; © Sj, that 


are defined component-wise: 


ML(T); := {#; € T; : md; € ACT) wi > ti}, 
(T)i:= {h € Ty: aay € A(S;) Wi > tit, 
PL(T); := {t; € Tj : at; € T; t} >7 tj},and 
PG(T); := {t; € Tj : 75s; € Sj 5; >r7 t;}. 


MLS, MGS, PLS, PGS are defined analogously with >> r instead of >r. 
For all T we have the following obvious inclusions: 


e Every M-operator eliminates more strategies than the corresponding 
P-operator. 

e Every operator considering (weak) dominance eliminates more 
strategies than the corresponding operator considering strict domi- 
nance. 

¢ With dominance in global games more strategies are eliminated 


than with dominance in local games. 
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PL(T) PLS(T) 
xy x 
ML(T) MLS(T) 
PG(T) i PGS(T) 
Zo Ai 
MG(T) MGS(T) 


Figure 4.1. Inclusions between the eight strategy elimination operators 


Each of these operators is deflationary, ie. F(T) C T for every T 
and every operator F. We iterate an operator beginning with T = S, 
ie. F°:= S and F**! := F(F*). Obviously, F° > F1D--.D F* D F**!, 
Since S is finite, we will reach a fixed point F“ such that F* = F a+]. FO 
We expect that for the eight fixed points MG®, ML, etc. the same 
inclusions hold as for the operators MG(T), ML(T), etc. But this is not 
the case: For the following game I = ({0,1}, (So, 51), (po, p1)) we have 
ML” ¢ PL”. 


a 2 
A | (2,1) (0,1) (1,0) 
B| (0,1) (2,1) (1,0) 
C | (1,1) (1,0) (0,0) 
D| (1,0) (0,1) (0,0) 


We have: 


¢ Z is dominated by X and Y. 
¢ D is dominated by A. 
° C is dominated by 5A + 5B. 


Thus: 


ML(S) = ML! = ({A,B},{X,Y}) Cc PL(S) = PL! 
— ({A,B,C},{X,Y}). 


ML(ML') = ML! since in the following game there are no domi- 
nated strategies: 
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PL(PL") = ({A,B,C},{X}) = PL? & PL! since Y is dominated by 
X (here we need the presence of C). Since B and C are now dominated 
by A, we have PL? = ({A}, {X}) = PL®. Thus, PL® & ML® although 
ML is the stronger operator. 

We are interested in the inclusions of the fixed points of the different 
operators. But we only know the inclusions for the operators. So 
the question arises under which assumptions can we prove, for two 
deflationary operators F and G on S, the following claim: 


If F(T) C G(T) for all T, then F° C G®? 


The obvious proof strategy is induction over a: We have F° = G° = S, 
and if F* C G*, then 


If we can show one of the inclusions F(F*) C F(G") or G(F*) C 
G(G*), then we have proven the claim. These inclusions hold if the 
operators are monotone: H : S — S is monotone if T C T’ implies 
H(T) C H(T’). Thus, we have shown: 


Lemma 4.18. Let F,G : P(S) — P(S) be two deflationary operators 
such that F(T) C G(T) for all T C S. If either F or G is monotone, then 
Pe Gr: 
Corollary 4.19. PL and ML are not monotone on every game. 

Which operators are monotone? Obviously, MGS and PGS are 
monotone: If ji; >7 s; and T’ C T, then also pi; > 5;. Let T’ C T and 
s; € PGS(T’);. Thus, there is no yj; € S; such that py; >> s;, and there 


is also no pj; € S; such that py; >>r s; and we have s; € PGS(T);. The 
reasoning for MGS is analogous if we replace S; by A(S;). 
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MLS and PLS are not monotone. Consider the following simple 


game: 


MLS({A, B}, {X}) = PLS({A, B}, {X}) = ({A}, {X}) and 
MLS({B}, {X}) = PLS({B}, {X}) = ({B}, {X}), 


but ({B},{X}) Z ({A}, {X))- 

Thus, none of the local operators (those which only consider domi- 
nant strategies in the current subgame) is monotone. We will see that 
also MG and PG are not monotone in general. The monotonicity of 
the global operators MGS and PGS will allow us to prove the expected 
inclusions ML® C MLS® C PLS® and PL® C PLS™ between the local 
operators. To this end, we will show that the fixed points of the local 
and corresponding global operators coincide (although the operators are 
different). 


Lemma 4.20. MGS” = MLS® and PGS” = PLS®. 


Proof. We will only prove PGS® = PLS®. Since PGS(T) C PLS(T) for 
all T and PGS is monotone, we have PGS® C PLS®. Now we will 
prove by induction that PLS“ C PGS" for all «. Only the induction step 
«++ «+1 has to be considered: Let s; € PLS? tt, Therefore, s; € PLS? 
and there is no si € PLS? such that s/ >>p,g« s;. Assume s; ¢ PGS?*1, ice. 


— {s, eon si > pcs si} gD 


(note: By induction hypothesis PGS* = PLS"). Pick an s* € A which 
is maximal with respect to >>prg«. Claim: s* € PLS*. Otherwise, there 
exists a B < a and an sj € S; with s’ > pgp Six. Since PLS? D> PLS*, 
it follows that si >>pyg« s* >>prsg« s;. Therefore, s; € A and s* is not 
maximal with respect to >>pys« in A. Contradiction. 

But if s* € PLS* and s* >>pyge sj, then s; ¢ PLS**! which again 


constitutes a contradiction. 
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The reasoning for MGS® and MLS® is analogous. Q.E.D. 
Corollary 4.21. MLS® C PLS”. 
Lemma 4.22. MG™ = ML” and PG® = PL”. 


Proof. We will only prove PG~ = PL” by proving PG* = PL* for all « 
by induction. Let PG* = PL“ and s; € PG**!. Then s; € PG? = PL* and 
hence there is no s/ € S; such that s/ >pcx s;. Thus, there is no si € PL# 
such that s} >pr« s; and s; € PL**!. So, PG**? C PL**1. 

Now, let s; € PLE Again we have s; © PL* = PG. Assume 
s; € Ports Then 


AS {s! € S;: 8; > pL s;} -w. 


For every B < «let AP = AN PLP . Pick the maximal B such that AP 4 © 
and a s* € AP which is maximal with respect to > pip 

Claim: 6 = a. Otherwise, s* ¢ PL? +! Then there exists an si € PL? 
with si >p,6 8}. Since PLP > PL” and s* >py« s;, we have si >pys s;, Le. 
s' € AP which contradicts the choice of s*. Therefore, s* € PL*. Since 
s* >pps sj, we have s; ¢ PL Contradiction, hence the assumption 
is wrong, and we have s; € PG**?. Altogether PG" = PL”. Again, the 


reasoning for MG® = ML” is analogous. Q.E.D. 
Corollary 4.23. PL® C PLS® and ML® C MLS®. 


Proof. We have PL® = PG® C PGS® = PLS® where the inclusion 
PG® C PGS” holds because PG(T) C PGS(T) for any T and PGS is 
monotone. Analogously, we have ML° = MG® C MGS® = MLS”. 

Q.E.D. 


This implies that MG and PG cannot be monotone. Otherwise, we 
would have ML® = PL®. But we know that this is wrong. 


4.6 Beliefs and Rationalisability 


Let T = (N, (Si)ien, (pi)icen) be a game. A belief of Player i is a probabil- 
ity distribution over S_j. 
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Remark 4.24. A belief is not necessarily a product of independent proba- 
bility distributions over the individual S; (j # 7). A player may believe 
that the other players play correlated. 


A strategy s; € S; is called a best response to a belief y € A(S_;) if 
Pilsi,¥) = pilsi, 7) for all si € S;. Conversely, s; € S; is never a best 
response if s; is not a best response for any y € A(S_j). 


Lemma 4.25. For every game T = (N, (S;)ien, (pi)ien) and every s; € Sj, 
s; is never a best response if and only if there exists a mixed strategy 
u; € A(S;) such that p; >s $;. 


Proof. If wi >s si, then pi(pi,S_i) > pi(si,s_i) for all s_; € S_;. Thus, 
pilvi, Y) > pilsi, Y) for all y € A(S_;). Then, for every belief 7 € A(S_j), 
there exists an si € supp(ju;) such that p;(s!, 7) > pi(si, 7). Therefore, s; 
is never a best response. 

Conversely, let s7 € S; be never a best response in I’. We define a 
two-person zero-sum game I” = ({0,1}, (To, 71), (p, —p)) where Ty = 
Sp={s7}, Tj, = S2; and p (636-4) = pilsizs1) — Pils ys4)~ Sinte:s* is 
never a best response, for every mixed strategy #4, € A(T,) = A(S_;) 
there is a strategy so € Ty = S; — {s7} such that pj(so,u1) > pi(s?, M1) 
(in T), i.e. p(so, 41) > 0 Gin I’). So, in I’ 


min max p(so, #11) > 0, 
meaty) se 7° P 


and therefore 


ae eaee , > 0. 
4 €A(T}) Ho A(To) P(Ho, #1) 


By Nash’s Theorem, there is a Nash equilibrium (p15, vj) in I’. By von 


Neumann and Morgenstern we have 


min max ’ _ * y 
eae M1) = P(Ho, Fi) 


= max min ; > 0. 
sp€A(To) 11 €A(T1) (Ho, H1) 


Thus, 0 < p(ui,HI) < plug, H,) for all wy € A(T,) = A(S_j). So, 
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we have in T pj(ug,s_,;) > pi(s?,s_;) for all s_; € S_; which means 
ue >> s*. Q.E.D. 


Definition 4.26. Let T = (N,(Sj)ien, (pi)ien) be a game. A strategy 
8; € 5; is rationalisable in T if for any Player j there exists a set Tj C 5S; 
such that 


es; € T;, and 
* every s; € T; (for all j) is a best response to a belief yj ¢ A(S_j) 
where supp(¥;) © T_};. 


Theorem 4.27. For every finite game I we have: s; is rationalisable if 
and only if s; € MLS;°. This means, the rationalisable strategies are 
exactly those surviving iterated elimination of strategies that are strictly 
dominated by mixed strategies. 


Proof. Let s; € S; be rationalisable by T = (Tj,...,Tn). We show T C 
MLS®. We will use the monotonicity of MGS and the fact that MLS® = 
MGS”. This implies MGS® = gfp(MGS) and hence, MGS® contains 
all other fixed points. It remains to show that MGS(T) = T. Every 
s; € Tj is a best response (among the strategies in S;) to a belief 7 
with supp(7) © T_;. This means that there exists no mixed strategy 
pj € A(S;) such that 4; >r s;. Therefore, s; is not eliminated by MGS: 
MGS(T) = T. 

Conversely, we have to show that every strategy s; € MLS? is 
rationalisable by MLS®. Since MLS® = MGS, we have MGS(MLS*) 
MLS®. Thus, for every s; € MLS? there is no mixed strategy J; 


an) 


A(S;) such that 1; >>yq,s~ sj. So, s; is a best response to a belief in 
MLS>°. Q.E.D. 


Intuitively, the concept of rationalisability is based on the idea that 
every player keeps those strategies that are a best response to a possible 
combined rational action of his opponents. As the following example 
shows, it is essential to also consider correlated actions of the players. 


Example 4.28. Consider the following cooperative game in which every 
player receives the same payoff: 


96 


4.6 Beliefs and Rationalisability 


Matrix 2 is not strictly dominated. Otherwise there were p,q € [0,1] 
with p+g <1 and 


8-p+3-(1—p-—q) >4and 
8-q+3-(1—p-—q) >4. 


This implies 2-(p+q)+6> 8,ie.2-(p+q) > 2, which is impossible. 


So, matrix 2 must be a best response to a belief y € A({T,B} x 
{L,R}). Indeed, the best responses to y = 5-((T,L) +(B,R)) are 
matrices 1, 2 or 3. 


On the other hand, matrix 2 is not a best response to a belief of 
independent actions y € A({T, B}) x A({L, R}). Otherwise, if matrix 2 
were a best response to y = (p-T+(1—p)-B,q-L+(1—q)-R), we 
would have that 


4pg+4-(1—p)-(1—q) = max{8pq,8- (1 — p)-(1—q),3}. 


We can simplify the left side: 4pq+4-(1—p)-(1-—q) = 8pq —4p —4q + 
4. Obviously, this term has to be greater than each of the terms from 
which we chose the maximum: 


8pq —4p —4g +4 > 8pq>pt+q>1 
and 


8pq —4p —4qg+4>8-(1—-p)-(1-q) > pt q <1. 


So we have p+ q = 1, org = 1~— p. But this allows us to substitute q by 
1 — p, and we get 


8pq — 4p —4q +4 = 8p- (1—p). 


However, this term must still be greater or equal than 3, so we get 
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8p-(l1—p) =3 


3 
a ae) 


which is impossible since max(p- (1 — p)) = } (see Figure 4.2). 


phot 
Net 
too 
ray 
— 


Figure 4.2. Graph of the function p+> p- (1— p) 


4.7 Games in Extensive Form 


A game in extensive form (with perfect information) is described by a 
game tree. For two-person games this is a special case of the games on 
graphs which we considered in the earlier chapters. The generalisation 
to n-person games is obvious: G = (V,V1,...,Vn,E,pi,---,Pn) where 
(V,E) is a directed tree (with root node w), V = V, ¥---WV,, and the 
payoff function p; : Plays(G) — IR for Player i, where Plays(G) is the 
set of paths through (V,E) beginning in the root node, which are either 
infinite or end in a terminal node. 

A strategy for Player i in G is a function f : {v € Vi: vE AO} 4 V 
such that f(v) € vE. S; is the set of all strategies for Player i. If all 
players 1,...,n each fix a strategy f; € S;, then this defines a unique play 
fi’: ++" fn © Plays(GQ). 

We say that G has finite horizon if the depth of the game tree (the 
length of the plays) is finite. 

For every game G in extensive form, we can construct a game 
S(G) = (N, (Siien, (Piien) with N = {1,...,n} and pi(fi,-.-, fn) = 
pilfi:--*fn). Hence, we can apply all solution concepts for strategic 
games (Nash equilibria, iterated elimination of dominated strategies, 
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etc.) to games in extensive form. First, we will discuss Nash equilibria 


in extensive games. 


Example 4.29. Consider the game G (of finite horizon) depicted in Fig- 
ure 4.3 presented as (a) an extensive-form game and as (b) a strategic- 
form game. The game has two Nash equilibria: 


e The natural solution (b,d) where both players win. 
e The second solution (a,c) which seems to be irrational since both 
players pick an action with which they lose. 


What seems irrational about the second solution is the following ob- 
servation. If Player 0 picks a, it does not matter which strategy her 
opponent chooses since the position v is never reached. Certainly, if 
Player 0 switches from a to b, and Player 1 still responds with c, the 
payoff of Player 0 does not increase. But this threat is not credible since 
if v is reached after action a, then action d is better for Player 1 than c. 
Hence, Player 0 has an incentive to switch from a to b. 


a 
(0,1) fo ee: od 
./ \i @IG). (01) 
b| (0,0) (1,1) 
(0,0) (1,1) 
(a) extensive form (b) strategic form 


Figure 4.3. A game of finite horizon 


This example shows that the solution concept of Nash equilibria 
is not sufficient for games in extensive form since they do not take the 
sequential structure into account. Before we introduce a stronger notion 
of equilibrium, we will need some more notation: Let G be a game in 
extensive form and v a position of G. G |, denotes the subgame of G 
beginning in v (defined by the subtree of G rooted at v). Payoffs: Let 
hy be the unique path from w to v in G. Then p?'° (70) = p9 (hp + 7). For 
every strategy f of Player i in G let f |p be the restriction of f to G |». 
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Definition 4.30. A subgame perfect equilibrium of G is a strategy profile 
(fi,.--,fn) such that, for every position v, (fi [v,..-,fn fo) is a Nash 
equilibrium of G |y. In particular, (f,,..., fn) itself is a Nash equilibrium. 


In the example above, only the natural solution (b,d) is a subgame 
perfect equilibrium. The second Nash equilibrium (a,c) is not a subgame 
perfect equilibrium since (a [y,c |v) is not a Nash equilibrium in G |». 

Let G be a game in extensive form, f = (f1,...,fn) be a strategy 


profile, and v a position in G. We denote by f(v) the play in G |, that is 
uniquely determined by f,..., fn. 


Lemma 4.31. Let G be a game in extensive form with finite horizon. 
A strategy profile f = (f1,..., fn) is a subgame perfect equilibrium of 
G if and only if for every Player i, every v € Vj, and every w € VE: 


pi(f(v)) = pi(f(w)). 

Proof. Let f be a subgame perfect equilibrium. If p;(f(w)) > pi(f(v)) 
for some v € Vj, w € vE, then it would be better for Player i in G |y to 
change her strategy in v from f; to f/ with 


ae e ifu Av 


w ifu=w. 


This is a contradiction. 

Conversely, if f is not a subgame perfect equilibrium, then there is a 
Player i, a position vo € V; and a strategy f/ # f; such that it is better for 
Player i in G |, to switch from fj to f/ against f_;. Let g:= (f/, f_i). We 
have q := pi(&(vo)) > pi(f(vo)). We consider the path (vo) = vo...0; 
and pick a maximal m < t with p;(&(v9)) > pi(f(vm)). Choose v = 0m 


and w = 0m+1 € VE. Claim: p;(f(v)) < pi(f(w)) (see Figure 4.4): 


pi(f(o)) = pilf(m)) < pi(F(m)) = 4 


pilf (w)) = pilf(Om+1)) 2 Pi(S(m+1)) = 9 QED. 


If f is not a subgame perfect equilibrium, then we find a subgame 
G |» such that there is a profitable deviation from fj in G [y, which only 
differs from fj in the first move. 
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Figure 4.4. p;(f(v)) < pi(f(w)) 


In extensive games with finite horizon we can directly define the 
payoff at the terminal nodes (the leaves of the game tree). We obtain a 
payoff function p; : T > R fori =1,...,n where T = {v € V: vE = ©}. 

Backwards induction: For finite games in extensive form we define 
a strategy profile f = (f1,..., fn) and values u;(v) for all positions v and 
every Player i by backwards induction: 


e For terminal nodes t € T we do not need to define f, and u;(t) := 
pi(t). 

e Let v € V \ T such that all u;(w) for all i and all w € vE are already 
defined. For i with v € V; define f;(v) = w for some w with 
uj(w) = max{u;(w’) : w’ € vE} and u;(v) := uj(fi(v)) for all j. 


We have p;(f(v)) = uj(v) for every i and every v. 


Theorem 4.32. The strategy profile defined by backwards induction is a 
subgame perfect equilibrium. 


Proof. Let f/ # fj. Then there is a node v9 € V; with minimal height 
in the game tree such that f/(v) A fi(v). Especially, for every w € vE, 


(ft, fi) (w) = Fw). For w = fi(v) we have 
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Therefore, f |, is a Nash equilibrium in G |>. Q.E.D. 


Corollary 4.33. Every finite game in extensive form has a subgame 
perfect equilibrium (and thus a Nash equilibrium) in pure strategies. 


4.8 Subgame-perfect equilibria in infinite games 


We now consider cases of infinite games in extensive form, for which 
we can establish the existence of subgame-perfect equilibria. General- 
izing the model of infinite two-person zero-sum games on graphs, we 
consider multi-player, turn-based games on graphs with arbitrary (not 
necesssarily antagonistic) qualitative objectives. 


Definition 4.34. An infinite (turn-based, qualitative) multiplayer game is 
a tuple G = (N,V, (V,)ien, E,Q, (Winj)icen) where N is a finite set of 
players, (V,E) is a (finite or infinite) directed graph (V;)jcn is a partition 
of V into the position sets for each player, OQ : V — C is a colouring 
of the positions by some finite set C of colours, and Win; C C“ is the 


winning condition for Player 7. 


For the sake of simplicity, we assume that uE := {v € V: (u,v) € 
E} # @ for all u € V, ie. each vertex of G has at least one outgoing 
edge. We call G a zero-sum game if the sets Win; define a partition of C”. 

A play of G is an infinite path through the graph (V,E), and a 
history is a finite initial segment of a play. We say that a play 7z is 
won by Player i if O(7t) € Winj;. A (pure) strategy of Player i in G is 
a function f : V*V; — V assigning to each sequence xv ending in a 
position v of Player i a next position f(xv) € vE. We say that a play 
m = 7(0)7(1)... of G is consistent with a strategy f of Player i if 
m(k+1) = f(7(0)...7(k)) for all k < w with 7(k) € Vj. A strategy 
profile of G is a tuple (f;);cen where f; is a strategy of Player i. 

It is sometimes convenient to designate an initial vertex vg € V 
of the game. We call the tuple (G,vo) an initialized infinite multiplayer 
game. A play (history) of (G,vo) is a play (history) of G starting with vo. 
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A strategy (strategy profile) of (G,vo) is just a strategy (strategy profile) 
of G. A strategy f of some player i in (G, vo) is winning if every play of 
(G, v9) consistent with o is won by player i. A strategy profile (fj) jen 
of (G,v9) determines a unique play of (G, vo) consistent with each fj, 
called the outcome of (f;)icn and denoted by ((f;)icn) or, in the case that 
the initial vertex is not understood from the context, ((fj)ien)v- In the 
following we will often use the term game to denote an (initialized) infinite 
multiplayer game according to Definition 4.34. 

For turn-based (non-stochastic) games with qualitative winning 
conditions, mixed strategies play no relevant role. Nash equilibria in 
pure strategies take the following form: 

A strategy profile (f;);cn of a game (G,v9) is a Nash equilibrium if 
for every player i and all her possible strategies f/ in (G,v9) the play 
(fi, Fi) jen\¢i}) is won by player i only if the play ((f;)jen) is also won 
by her. 

Despite the importance and popularity of Nash equilibria, there are 
several problems with this solution concept, in particular for games that 
extend over time. This is due to the fact that Nash equilibria do not take 
into account the sequential nature of games and all the consequences of 
this. After any initial segment of a play, the players face a new situation 
and may change their strategies. Choices made because of a threat by 
the other players may no longer be rational, because the opponents have 
lost their power of retaliation in the remaining play. 


Example 4.35. Consider a two-player Biichi game with its arena depicted 
in Figure 4.5; round vertices are controlled by player 1; boxed vertices are 
controlled by player 2; both players win if and only if vertex 3 is visited 
(infinitely often); the initial vertex is 1. Intuitively, the only rational 
outcome of this game should be the play 123“. However, the game has 
two Nash equilibria: 


(1) Player 1 moves from vertex 1 to vertex 2, and player 2 moves from 
vertex 2 to vertex 3. Hence, both players win. 

(2) Player 1 moves from vertex 1 to vertex 4, and player 2 moves from 
vertex 2 to vertex 5. Both players lose. 


The second equilibrium certainly does not describe a rational be- 
haviour. Indeed both players move according to a strategy that is always 
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losing (whatever the other player does), and once player 1 has moved 
from vertex 1 to vertex 2, then the rational behaviour of player 2 would 
be to change her strategy and move to vertex 3 instead of vertex 5 as this 
is then the only way for her to win. 


.. 


5 


Figure 4.5. A two-player Biichi game. 


This example can be modified in many ways. Indeed we can con- 
struct games with Nash equilibria in which every players moves infinitely 
often according to a losing strategy, and only has a chance to win if she 
deviates from the equilibrium strategy. The following is an instructive 
example with quantitative objectives. 


Example 4.36. Let G, be an n-player game with positions 0,...,n — 1. 
Position n is the initial position, and position 0 is the terminal position. 
Player i moves at position i and has two options. Either she loops at 
position 7 (and stays in control) or moves to position i — 1 (handing 
control to the next player). For each player, the value of a play 7 is 
n/|7|. Hence, for all players, the shortest possible play has value 1, 
and all infinite plays have value 0. Obviously, the rational behaviour 
for each player 7 is to move from i to i — 1. This strategy profile, which 
is of course a Nash equilibrium, gives value 1 to all players. However, 
the ‘most stupid’ strategy profile, where each player loops forever at his 
position, i.e. moves forever according to a losing strategy, is also a Nash 
equilibrium. 

For a game G = (N,V, (V)ien, E,O, (Win;)jer) and a history h of 
G, let the game G|;, = (N,V, (Vj)ien, E,.O, (Wini|n) cen) be defined by 
Winj|, = {a € C® : O(h)-« © Win;}. For an initialized game (G, v9) 
and a history hv of (G,v9), we call the initialized game (G|),,v) the 
subgame of (G,v9) with history hv. For a strategy f of Player i in G, let 
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fln : V*V; — V be defined by f|,(xv) := f(hxv). Obviously, f |p, is a 
strategy of Player i in G|p. 

Recall that a strategy profile (f;)ien is a subgame perfect equilibrium 
(SPE) if (filn)icen is a Nash equilibrium of (G|,,v) for every history hv 
of (G, v0). 


Example 4.37. Consider again the game described in Example 4.35. The 
Nash equilibrium where Player 1 moves from vertex 1 to vertex 4 and 
Player 2 moves from vertex 2 to vertex 5 is not a subgame perfect 
equilibrium since moving from vertex 2 to vertex 5 is not optimal for 
Player 2 after the play has reached vertex 2. On the other hand, the Nash 
equilibrium where Player 1 moves from vertex 1 to vertex 2 and Player 2 
moves from vertex 2 to vertex 3 is also a subgame perfect equilibrium. 


The first step in the analysis of subgame perfect equilibria for infinite 
duration games is the notion of subgame-perfect determinacy. While the 
notion of subgame perfect equilibrium makes sense for more general 
classes of extensive games, the notion of subgame-perfect determinacy 
applies only to games with qualitative winning conditions. 


Definition 4.38. A game (G,vo) is subgame-perfect determined if there 
exists a strategy profile (f;)j-n) such that for each history hv of the game 
one of the strategies f;|, is a winning strategy in (G|p,,7). 


Proposition 4.39. Let (G,v9) be a qualitative zero-sum game such that 
every subgame is determined. Then (G,v9) is subgame-perfect deter- 
mined. 


Proof. Let (G,vo) be a multiplayer game such that, for every history hv, 
there exists a strategy f! for some player i, which is winning in (G|),,7). 
We have to combine these strategies in an appropriate way to strategies 
fi. (Let us point out that the trivial combination, namely f;(iv) := f!(v) 
does not work in general.) We say that a decomposition h = hy - hz is 
good for player i w.r.t. vertex v if fi" h, is winning in (G|;,v). If the 
strategy f!' is winning in (G|j,,0), then the decomposition h = h-e is 
good w.r.t. v, so a good decomposition exists. 


For each history hv, if f! is winning in (G|),,7), we choose the good 
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(w.r.t. vertex v) decomposition h = hyh2 with minimal h,, and put 


fi(hv) := fi" (hyo). 


Otherwise, we set f;(hv) := f!'(v). 

It remains to show that for each history hv of (G,v9) the strategy 
fily is winning in (G|,,0) whenever the strategy f!’ is. Hence, assume 
that f!’ is winning in (G|,,0), and let 77 = 7(0)7t(1)... be a play starting 
in 71(0) = v and consistent with f;|;, We need to show that 7 is won by 
player iin (G|;,,v). 

First, we claim that for each k < w there exists a decomposition 
of the form h7r(0)...7(k—1) = hy - (hg7t(0)... a (k —1)) that is good 
for player i w.rt. 7(k). This is obviously true for k = 0. Now, for 
k > 0, assume that there exists a decomposition h7(0)...7(k —2) = 
hy - (hg7t(0)...7(k — 2)) that is good for player i wrt. 7(k — 1) 
and with h, being minimal. Then z(k) = fj(h7(0)...7a(k—1)) = 
f!™(hg7(0)...7(k —1), and h7c(0) ... (kK —1) = hy(hz70(0) ... 0(k —1)) 
is a decomposition that is good w.r.t. 7¢(k). 

Now consider the sequence h?, ht,... of prefixes of the good decom- 
positions h7r(0)...7(k —1) = WkhKre(0)...7(k —1) (wart. 7(k)) with 
each hk being minimal. Then we have ho = ht > ..., since for each k > 0 
the decomposition /7(0) ... 77(k — 1) = hk "h&-'7(0) ... 21(k — 1) is also 
good for player i w.r.t. 7(k). As < is well-founded, there must exist 
k < w such that hy := hk = hi for each k < 1 < w. Hence, we have that 
the play 7(k)7(k +1)... is consistent with i lio7(0)...7(k—1), Which is 
a winning strategy in (G|j2(0)...2(k—1), 71(k)). So the play hz is won by 
player i in (G, v9), which implies that the play 7 is won by player i in 
(G\n,0). Q.E.D. 


We say that a class of winning conditions is closed under taking 
subgames, if for every condition X C C” in the class, and every h € C’, 
also X|;, := {x € C” : hx € X} belongs to the class. Since Borel winning 
conditions are closed under taking subgames, it follows that any two- 
player zero-sum game with Borel winning condition is subgame-perfect 
determined. 
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Corollary 4.40. Let (G,vo) be a two-player zero-sum Borel game. Then 
(G, v9) is subgame-perfect determined. 


Multiplayer games are usually not zero-sum games. Indeed when 
we have many players the assumption that the winning conditions 
of the players form a partition of the set of plays is very restrictive 
and unnatural. We now drop this assumption and establish general 
conditions under which a multiplayer game admits a subgame perfect 
equilibrium. In fact we will relate the existence of subgame perfect 
equilibria with the determinacy of associated two-player games. In 
particular, it will follow that every multiplayer game with Borel winning 
conditions has a subgame perfect equilibrium. 

In the rest of this subsection, we are only concerned with the existence 
of equilibria, not with their complexity. Thus, without loss of generality, 
we tacitly assume that the arena of the game under consideration is a tree 
or a forest with the initial vertex as one of its root. The justification for 
this assumption is that we can always replace the arena of an arbitrary 
game by its unravelling from the initial vertex, ending up in an equivalent 


game. 


Definition 4.41. Let G = (N,V,(Vi)ien, E,Q, (Win;)icn) be a multi- 
player game (played on a forest), with winning conditions Win; € C”. 
The associated class Two(G) of two-player zero-sum games is obtained 
as follows: 


(1) For each player i, Two(G) contains the game G; where player i plays 
G, with his winning condition Win,, against the coalition of all other 
players, with winning condition C” \ Win,. 

(2) Close the class under taking subgames (i.e. consider plays after 
initial histories). 

(3) Close the class under taking subgraphs (i.e. admit deletion of 


positions and moves). 


Note that the order in which the operations (1), (2), and (3) are 
applied has no effect on the class Two(G). 


Theorem 4.42. Let (G, v9) be a multiplayer game such that every game in 
Two(G) is determined. Then (G,vo) has a subgame perfect equilibrium. 
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Proof. Let G = (N,V, (Vi)ien, E,O, (Win;)ien) be a multiplayer game 
such that every game in Two(G) is determined. For each ordinal « we 
define a set E* C E beginning with E° = E and 


poe (Ee 


a<a 


for limit ordinals A. To define E*+! from E", we consider for each 
player i € N the two-player zero-sum game G* = (V, Vj, E*,O, Win;) 
where player i plays, with his winning condition Win; against the coali- 
tion of all other players (with winning condition C” \ Win;). Every 
subgame of G* belongs to Two(G) and is therefore determined. Hence 
we can use Proposition 4.39 to fix a subgame perfect equilibrium (f?, f*;) 
of (G, vo) where f* is a strategy of player i and f"; is a strategy of the 
coalition. Moreover, as the arena of G* is a forest, these strategies can 
be assumed to be positional. Let X* be the set of all v € V such that f? 
is winning in (G*|,,v) for the unique maximal history h of G leading to 
v. For vertices v € V;M Xj we delete all outgoing edges except the one 
taken by the strategy f?, i.e. we define 


Ett) = E*\ (J{(u,v) © E: u € ViN Xf and v F f"(u)}. 
i¢N 
Obviously, the sequence (E*),con is non-increasing. Thus we can fix 
the least ordinal y with E’ = EY*! and define f; = f,’ and f_; = f’,. 
Moreover, for each player j # i let f;; be the positional strategy of 
player j in G that is induced by f_j;. 


Intuitively, Player i’s equilibrium strategy 9; is as follows: Player 
i plays f; as long as no other player deviates. Whenever some player 
j #i deviates from her equilibrium strategy fj, player i switches to fj. 
Formally, define for each vertex v € V the player p(v) who has to be 
“punished” at vertex v where p(v) = L if nobody has to be punished. If 
the game has just started, no player should be punished. Thus we let 


p(v) = Lif vis a root. 


At vertex v with predecessor u, the same player has to be punished as 
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at vertex u as long as the player whose turn it was at vertex u did not 
deviate from her prescribed strategy. Thus for u € V; and v € uE we let 


ali if p(u) = L and v = f;(u), 
pe) = 4 plu) if p(u) Ai, pu) A Land v= fi pu)(u), 


1 otherwise. 


Now, for each player 7 € N we can define the equilibrium strategy 9; by 
setting 

Soe oa if p(v) = L or p(v) =i, 

fip(v)(v) otherwise 
for each v € V. 

It remains to show that (g;)icn is a subgame perfect equilibrium 
of (G, v9). First note that f; is winning in (G."|;,,v) if f* is winning in 
(G*|,,0) for some ordinal a because if f* is winning in (G#|;,,v) every 
play of (G;+"|),,) is consistent with f* and therefore won by player i. 
As EY C E**1, this also holds for every play of (G."|;,,). Now let v be 
any vertex of G with h the unique maximal history of G leading to v. 
We claim that (gj) jen is a Nash equilibrium of (G|},,v). Towards this, let 
g’ be any strategy of any player i € N in G; let 7 = ((g;)jen)v, and let 
m' = (8", (j)jen\{i})o- We show that hz is won by player i or that hzt’ 
is not won by player i. The claim is trivial if 7 = 7’. Thus assume that 
m # 7’ and fix the least k < w such that 1(k +1) 4 7'(k +1). Clearly, 
m(k) € V; and g’(m(k)) 4 g;(7(k)). Without loss of generality, let k = 0. 
We distinguish the following two cases: 


¢ f; is winning in (G;|),,7). By the definition of each gj, 71 is a play 
of (G.’|,,v). We claim that 7 is consistent with f;, which implies 
that h7t is won by player i. Otherwise fix the least | < w such that 
m(1) € V, and fj(7(1)) A m(1+1). As fj is winning in (G."|j,,0), fi is 
also winning in (Gi lnn(0)...e(1-1)» m(1)). But then (7r(1), 7(1+1)) € 
EY \ EY+!, a contradiction to EY = EY, 

° f; is not winning in (G,"|,,0). Hence f_; is winning in (G,"|y,0). 
As ¢'(v) 4 g;i(v), player i has deviated, and it is the case that 
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m' = (g', (fii)jen\{i})v- We claim that 71’ is a play of (G;’|n,0). As fi 
is winning in (G."|;,,v), this implies that ht’ is not won by player i. 
Otherwise fix the least 1 < w such that (7’'(1),7’'(1+1)) ¢ E7 
together with the ordinal w such that (7/(1), 7/(1+1)) € E* \ E&*1. 
Clearly, 7c'(I) € Vj. Thus f# is winning in (G?|pq(0)..0'(1—1), T (1), 
which implies that f; is winning in (G7 live!(0)...7°(1—-1)7 70 (1)). As 70! 
is consistent with f_;, this means that f_; is a winning in (G."|1,,2), 
a contradiction. 


It follows that (gj) jen = (gjln)jen is a Nash equilibrium of (G|,7) for 
every history hv of (G, v0), hence (g;)jen is a subgame perfect equilib- 
rium of (G, vo). Q.E.D. 


Corollary 4.43. Every multiplayer game with Borel winning conditions 
has a subgame perfect equilibrium. 


O course this also implies that every multiplayer game with Borel 
winning conditions has a Nash equilibrium. Indeed, for the existence 
of Nash equilibria, a slightly weaker condition suffices. Let Two(G) Nash 
be defined in the same way as Two(G) but without closure under sub- 
graphs. 


Corollary 4.44. If every game in Two(G)Nash is determined, then G has 
a Nash equilibrium. 
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The standard basic notion used in mathematics is the notion of a set, 
and all mathematical theorems follow from the axioms of set theory. The 
standard set of axioms, which (among others) guarantee the existence of 
an empty set, an infinite set, and the powerset of any set, and that no 
set is a member of itself (ie. Vx =x € x) is called the Zermelo-Friinkel Set 
Theory ZF. Furthermore, it is consequence of ZF that every set a contains 
an €-minimal element b € a, i.e. bM a = ©. This implies that there are 
no infinite 5-sequences x; > X72 5 x3 5 ..., because otherwise the set 
{x1,X2,X3,...} would not contain an €-minimal element. It is standard 
in mathematics to use ZF extended by the axiom of choice AC, which 
together are called ZFC. 

Since everything is a set in mathematics, there is a need to represent 
numbers as sets. The standard way to do this is to start with the empty 
set, let 0 = ©, and proceed by induction, defining n+ 1=nU {n}. Here 
are the first few numbers in this coding: 

°0=2, 

°1= {2}, 

© 2 ={D,{O}}, 

3 = {2 {DO} AD {Oh tt =2U {2}, 

©4= {DO} AD AOL AP AOE AD ADF rt = 30 13h. 


Observe that for each number n (as a set) it holds that 
men => m Cn for every set m. (4.1) 


Sets satisfying property (4.1) are called transitive sets, because (4.1) is 
equivalent to 


xeyen => x En for every set x,y. 


For example, the set a := {©,{@},{{O}}} F# 3 is a transitive set, 
but a does not occur on our list of natural numbers. Intuitively, the 
problem is that {{O}} € © and © ¢ {{O}}, so € is not trichotomous 
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on a. This is why, € does not constitute a linear order on a. Now, 
we define a more general class of numbers, the so-called von Neumann 
ordinal numbers. 


Definition 4.45. A set w is an ordinal if 


(1) @ is transitive, i.e. x Cy Cu => x € a for every x,y, and 
(2) € is trichotomous on 4, i.e. for every x,y € a either x = yorx cy 
ory € x. 


On := {a : w is an ordinal} is the class of all ordinals. 


We are going to prove in Theorem 4.47 that for all ordinal a, B it 
holds that either 


a=boracborad b. 


It is even the cases, that the class of ordinal numbers forms a well-founded 
order (w.r.t. €). This means, that € is a linear order on the class of 
ordinals and that every non-empty class X of ordinal number contains 
an €-minimal ordinal « € X, i.e. w € B for every B € X \ {a}. Note, that 
this also implies that the class On is a proper class, which means that 
On is not a set itself (otherwise On would satisfy Definition 4.45 and, 
hence, On € On in contradiction to the ZFC axioms). 

It is easy to check that the natural numbers we defined above are 
ordinal numbers: Indeed, if 1 is a natural number, then we have that 
n = {0,...,n—1} and, consequently, for every i € n follows that 
i= {0,...,d—1} C {0,...,i—1,i,...n -—1} =n. Similarly, it is easy to 
see that for every m,k € n that either m = k or m € k or k € m holds. It 
is worth mentioning that the relation € coincides with the usual order 
< on natural numbers. 

Except for natural numbers, are there any other ordinal numbers? 
In fact, we shall see that there are infinite many ordinals which are 
infinitely large. For example, consider w which is defined by 


w= (Jn =(G.0.% 1) ={0,1;2,3,.2:}. 


w is the set of all natural numbers, but it is easy to verify that it satisfies 
Definition 4.45 and, hence, w is also an ordinal number. But it does 
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not stop here! It is always possible to apply the +1 operation, which is 
defined as 


at+1:=avu {a}. 


Lemma 4.46. Let a be an ordinal and 6 € a. Then f and «+1 are 
ordinals as well. 


Proof. First, we prove that f is an ordinal. To do this, we need to prove 
that 6 satisfies (1) and (2) of Definition 4.45. 


(1) For this, let d € c € B. We need to show that d € B. Due to 

b €c € B € « and the transitivity of « (Definition 4.45 (1)), it 
follows that b,c € a. Thus, B,c,d € w. By Definition 4.45 (2), we can 
conclude that 6 = d or B € d ord € B holds. 
B =d is impossible, because 6 = d would implies that d €c € B =d 
and, thus,c 3d 3c 5... but due to the ZFC axioms there are no 
infinite 5-sequences. Similarly, 6 € d is also wrong since otherwise 
déece€f€d. Therefore, d € £ has to be true. 

(2) It remains to show that Definition 4.45 (2) is true for 6. But this is 
trivial because, due to Definition 4.45 (1), it is the case that B C a 
and condition (2) is assumed to be true for a. 


Now we demonstrate that « + 1 is an ordinal number. 


(1) Transitivity of a +1: Let c € b € a+1. Our goal is to prove that 
c€a+l1. Sincexa+1=aU {a}, we can distinguish the following 
two cases. If b = a, then c € b € a and, by using the transitivity of 
a, we can deduce that 


c€aCaU{e}=a+l. 


Otherwise, b € « and then b € a (because b € « + 1). By transitivity 
of a, we obtainc Ea Ca+l. 

(2) Trichotomy: Let x,y € a+1. We need to prove that x = y or 
x € yory € x holds. If both x,y € a, then there is nothing to 
prove. Hence, x ¢ a or y ¢ w. W.Lo.g. we assume that x ¢ a. Since 
x €a-+1, it follows that x = a. If y € a, then we are done. If, on the 
other hand, y ¢ a, then x = a = y. Thus, we obtain x € y or x = y. 
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Q.E.D. 


But does it make sense to say that w + 1 is the next ordinal, is there 
an order on ordinals? 


In fact, the ordinal numbers are linearly ordered by €. 


Theorem 4.47. For every ordinal a,6 either a = B ora € Bor B € a. 
Furthermore, a C 6 holds, if and only if a € B or a = B. 


Before can prove this theorem, we need some lemmas first. 


Lemma 4.48. If X is a non-empty class of ordinals, then 
()X := {x : x €a for every a € X} 


is an ordinal. 


Proof. Since X is non-empty, there is an ordinal a € X and, then, (1) X C «. 
Because a is a Set, it is possible to prove (by using the ZFC axioms) that 
(| X is a set. Now it suffices to prove that (| X satisfies the two conditions 
from Definition 4.45: 


(1) Transitivity: Leta € b © (|X. Thena <b € ¥ for all y € X. Since X 
is a class of ordinals, it follows that a € ¥y for all y € X and, finally, 
ae ()X. 

(2) Trichotomy: Let a,b € (|X. Then a,b € «a and, because « is an 
ordinal,a€ bora=borb€a. 


Q.E.D. 

The transitivity of ordinals allows us to prove that elements of 
ordinals are subsets. Of course, the converse is not true in general, 
because not every subset of an ordinal is an element. However, proper 


subsets that are ordinals turn out to be elements. As usual we write 
a« C Basashorthand for a C B and a F B. 


Lemma 4.49. Let a, 6 be ordinals and a C B. Then a € B. 


Proof. Towards a contradiction, we assume there are some ordinals « C B 
with a ¢ B. 
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In order to obtain a contradiction, we prove that there is an infinite 
>-sequence fo 5 fi 5 fo... of ordinals starting at 6 such that a C B; 
but « ¢ B; for alli € {0,1,2,...}. 

We start with Bo := 6. Now, consider the set 


Bo\«:= {y € Bo : yEa}. 


We define 7 := (\(fo \ «). Due to « C B = Bo, there is a pt € Bo \a. Asa 
result, Bo \ « # © and 7¥ is an ordinal (by Lemma 4.48). 


Claim 4.50. a C ¥. 


Proof. Let 6 € a. We are going to prove that 6 € 7. 
Since « C Bo we have 6 € Bo. Let pu! € Bo \ & be picked arbitrarily. 
As a result p’,6 € Bo and, by Definition 4.45 (2), it follows that 


w=dorp' €dordenp. 


We observe that pu’ 4 6, because pi’ ¢ « but 6 € w. Furthermore, 
u’ € 6, because otherwise p’ € 5 € « and (since « is an ordinal) py’ € a 
but pl’ € a. 

Therefore, it must be the case that 6 € py’. pu’ © Bo \ « was chosen 
arbitrarily, so 6 € (\(Bo \a) = 7. Q.E.D. 


Now we have 


PeEV= (1) 80 ea 


Recall that pz € Bo \ « and, therefore, y = (\(fo \ «) C pu. Together with 
a C ¥ this leads to a C yp. Since pv € Bo and a ¢ Bo, it follows that a C p. 
Furthermore, « ¢ p4, because otherwise a € pt € Bo and then a € Bo 
(because fo is an ordinal) in contradiction to a ¢ Bo. 

All in all, we managed to prove that  € Bo is an ordinal (due to 
Lemma 4.46) with a C p but a ¢ pw. Hence, we can set f = pl. 

By repetition, we can construct the desired sequence Bo > £1 > B2 > 
..., but this contradicts the ZFC axioms! 

Q.E.D. 


Now we have all the tools we need to finally prove Theorem 4.47. 
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Proof (of Theorem 4.47). First we prove that a CB —+a=BV«ae Bf. 
The direction “<—” follows intermediately from Definition 4.45 (1), 
while “=>” is Lemma 4.49. 
Now we demonstrate that € is a linear order on the class of ordinal 
numbers. Towards a contradiction, assume that there are ordinals a, 6 
that are incomparable w.r.t. €, ie., we have 


a#ABPanda€¢ Bf andfé€u. (4.2) 


Consider aM $6. By Lemma 4.48, «1 6 is an ordinal. Furthermore, 
aNBCaandans CB. Ifa =a 8, then a C B and by Lemma 4.49 
either « = 6 or a € f in contradiction to (4.2). Thus, a 4 «MB and, 
similarly, B A «1 B. 

But then, aM B C wand aM C B, which implies that aM B € a and 
a B € B, which leads to aM B € aM B, but due to the ZFC axioms this 
is not possible! Contradiction! 


So, € is in fact a linear order on the class of ordinal numbers. Q.E.D. 


Recall that On is the class of all ordinals. Theorem 4.47 tells us that 
€ is a linear order on On. More general, € is a well-founded order on On. 
An order (A, <) is a well-founded order, if 


(1) (A, <) is a linear order and 
(2) for every non-empty set X C A there is a <-minimal element x € X, 
i.e., x < y for every y € X. 


For example, (IN, <) is a well-founded order but (Z, <) or (Qs0, <) 
are not well-founded orders. 

It is not difficult to see that ordinal numbers are well-founded 
orders (w.r.t. €). Indeed, if X C On is a non-empty class of ordinals, 
then y := (|X is an ordinal (by Lemma 4.48) and y C x for all x € X. 
It remains to prove that y € X: Otherwise y ¢ X and then y C x for 
all x € X. This leads to y € x for all x € X (by Theorem 4.47). Thus, 
{y} © x and, as a consequence, y +1 C x for all x € X. But then 
y¥+1OCNX=7 = ¥€7 which violates the ZFC axioms! Hence, 
(he = ee. 

Now we turn our attention towards the construction of bigger 
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ordinals. For this, we need the following lemma which states that 


ordinal numbers are closed under unions. 


Lemma 4.51. Let x be a set of ordinals, i.e., every a € x is an ordinal. 
Then 


x := {B : B € a for some a € x} 
is an ordinal number. 


Proof. Using the ZFC axioms, it is possible to prove that Ux is a set. 
Hence, it remains to show that (1) and (2) of Definition 4.45 are satisfied. 


(1) Transitivity of Ux: If a € b € Ux, then there is ac € x such that 
a € b € cand, by transitivity, a € c which implies that a € Ux. 

(2) Trichotomy: If a,b © Ux. Then there are some c,d € x such that 
aecandbc€d. Applying Lemma 4.46 yields that a,b are ordinals 
and, by Theorem 4.47, either a = bora cborbe€a. 


Q.E.D. 


w := U,,n, the union of all natural numbers, is again an ordinal 
number. To prove this, we observe that w = Uw and use Lemma 4.51 
(that w is a set is a consequence of the axiom of infinity). 

What is the next ordinal number after w? We can again apply the 
+1 operation in the same way as for natural numbers, so 


wpa {we pa 101254071 2h bi 


Of course it is now possible to construction ordinals like w +2 := 
(w+1)+1,w+3,... and then we can build the union 


wtw=|(Jwti= {0,1,2,...,.~,wt+1w+2,...}, 


1EwW 


which is an ordinal because of Lemma 4.51. The fact that 
wU{w+i : i€w} is a set can be proven by using the axiom of re- 
placement. 
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To get an intuition on how ordinals look like, consider the following 
examples of infinite ordinals: w+1, w+w = 2w, 3w,...,w-w = 
w, w,..., Ww”. 

For some ordinals « it is the case that a = B + 1 for some f. However, 


it is not possible to find an ordinal y such that y + 1 = w holds (Why?). 


Definition 4.52. Let a ¢ 0 be an ordinal. If B+ 1 € a for every B € a, 
then we call « an limes ordinal. 


It is easy to see that A is an limes ordinal, if and only if A 4 0 and 
A= 

Ordinals that are not limes ordinals are called successor ordinals 
because of the following theorem. 


Theorem 4.53. Let « 4 0 be an ordinal that is not an limes ordinal. Then 
there is an ordinal 6 such that B +1 =a. 


Proof. By Definition 4.52 there is a B € « such that B+1 ¢ a. By Theorem 
4.47, either B+1—=aorBpB+15a. 

So, we only need to show that 8 +1 3% a holds. Otherwise a € 
Bb+1= BU {6}. Clearly, « ¢ {6} because « = 6 € « would violate the 
ZFC axioms. But then a € 6B € a which contradicts the ZFC axioms as 
well. Hence 6 + 1 5 a is impossible which leads to B +1 =a. Q.E.D. 


Ordinals are intimately connected to well-orders. In fact any well- 
ordering (A,<) is isomorphic to some («, €) where a is an ordinal. 
For example, (IN,<) is isomorphic to (w,€) and w +w represents 
({0,1} x IN, <jex) where <jex is the lexicographical order. 

The well-ordering of ordinals allows to define and prove the princi- 
ple of transfinite induction. This principle states that On, the class of all 
ordinals, is generated from © by taking the successor (+1) and the union 
on limit steps, as shown on the examples before. 

The principle of transfinite induction allows us to define sets Xq 
where « is an ordinal number. Since On is a well-order, we only need to 
describe how X, is constructed under the assumption that X¢ is already 
defined for every B € «a. 

For example, it is possible to define (via transfinite induction) the 
winning region of player 0 in a reachability game (V, Vo, Vi, E). To do 
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this, we define sets W? for every ordinal number a: 


We := ©, 
Wo, := {xe Vo xEOWe #ObU {xe Vy xE CW}, 


we — ie, Wp for limes ordinals 1. 
Bea 


Now it is easy to verify that U,con W? is the winning region of 
Player 0. 


4.9 Cardinal Numbers 


Besides ordinals, we sometimes need cardinal numbers which are special 
ordinal number that can be used to measure the size of sets. We say that 
two sets x, y have the same cardinality, if there is a bijection between x 
and y. 


Definition 4.54. An ordinal x is a cardinal number, if for every a € x there 
is no bijection between x and a. Furthermore, we say that a cardinal 
number x is the cardinality of a set x, if there is a bijection between x 
and x. In this case we let |x| := x. 

Cn := {x € On : x isa cardinal number} is the class of all cardinal 


numbers. 


But is it guaranteed that we really find a cardinal number for every 
possible set out there? The next theorem answers this question. 


Theorem 4.55. For every set x there is a cardinal number |x|. 
Proof. Consider the class Y of ordinals, which is given by 


Y := {w € On : there is a bijection f : x — a} 


= {a € On : there is a bijection f : a > x}. 


If Y is non-empty, then |x| := (| Y € Y is the desired cardinal number. 

Now we prove that X # © is indeed the case. By the axiom of 
choice, there is a choice function g for x, i.e., for every y C x with y A © 
we have g(y) € y. 
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Using transfinite induction, we define for every ordinal a an object 
Xq by 


sya) ifyq=x\{xp: PEat #@ 
x ifyy= 


Xn — 


It is easy to see that for every x, # x we have that x, is an element 
of x but x» # xg for every B € a. 

If x, = x holds for some ordinal «a, then there is a minimal ordinal 
a’ C w such that x, = x and, by definition of x, this means that 
x = {xg : B €a’}. Furthermore, the function f : a’ + x, B+ xgisa 
bijection between x and a’. This implies that a’ € Y. 

So, it only remains to prove that x, = x for some ordinal a. Towards 
a contradiction, we assume that x, 4 x for every ordinal w. Then every 
Xy € x and, therefore, the mapping f : On —> x! := {xy : a € On}, aH 
Xq is a bijection between On and x’. Since x is a set, x’ C x is a set as 
well. Therefore, by the axiom of replacement, 


> ele {f7) Dye x'} = On 


is a set. As a result, On satisfies Definition 4.45 and, consequently, 
On € On which violates the ZFC axioms! Contradiction! 


Q.E.D. 


It is worth mentioning that the enumeration (xa),c|,) induces a 
well-ordering < on x by 


Xan <Xg => ae Bp. 


Corollary 4.56 (Well-ordering theorem). Every set x can be well-ordered, 
i.e., there is a well-order < on x. 


Every finite ordinal number is a cardinal number but there are also 
infinite cardinal numbers. For example, I := w is the smallest infinite 
cardinal number and, by using the power set, we can construct strictly 
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larger cardinal numbers: 


a Dee |P (Ae), 
ee U jp for limes ordinals A. 
Bea 


Please observe that J) = |P(w)| = |R]. 

Whether there exists cardinal numbers between Ig and 2; is called 
the continuum hypothesis (CH) which has turned out to be independent 
of ZFC, i.e., neither (CH) nor —(CH) are consequences of ZFC. 
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